2024-04-25T20:02:06.018 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T20:02:06.018 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_teuthology@teuthology --name teuthology-2024-04-23_22:16:02-rbd-reef-distro-default-smithi --archive /home/teuthworker/archive/teuthology-2024-04-23_22:16:02-rbd-reef-distro-default-smithi/7670822 --verbose --description rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/journal} -- /home/teuthworker/archive/teuthology-2024-04-23_22:16:02-rbd-reef-distro-default-smithi/7670822/orig.config.yaml 2024-04-25T20:02:06.050 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T20:02:06.130 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-23_22:16:02-rbd-reef-distro-default-smithi/7670822 branch: reef description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/journal} email: ceph-qa@ceph.com first_in_suite: false job_id: '7670822' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-04-23_22:16:02-rbd-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: reef ceph: conf: global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: lz4 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from sha1: 646b8c7d293d8ec31a40c69f7b3237b19c1be24d ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 646b8c7d293d8ec31a40c69f7b3237b19c1be24d thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef sha1: 19b26365dac54ee709df8c5dc2a7b7e89a50aba5 owner: scheduled_teuthology@teuthology priority: 930 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 2987 sha1: 646b8c7d293d8ec31a40c69f7b3237b19c1be24d sleep_before_teardown: 0 subset: 17/128 suite: rbd suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_19b26365dac54ee709df8c5dc2a7b7e89a50aba5/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 19b26365dac54ee709df8c5dc2a7b7e89a50aba5 targets: smithi039.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPx7DNlgqpi1ZwRCwkvSCxi2BlSMk8UY6pu4oQo7xnMWVcb1sBPhUbx41iRbpN+5U6ckuXt/05601eWN/K2B+Jw= smithi135.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBONrHp0x4i8dM+RTeVczM1nKavKGfa39I1L1k+GaFeYF9LPF9lEqkUkFW7f1dKw8nW8rHjX+IdiF6GXTDpdianQ= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - workunit: clients: client.0: - rbd/journal.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-23_22:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T20:02:06.130 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_19b26365dac54ee709df8c5dc2a7b7e89a50aba5/qa; will attempt to use it 2024-04-25T20:02:06.131 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_19b26365dac54ee709df8c5dc2a7b7e89a50aba5/qa/tasks 2024-04-25T20:02:06.131 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T20:02:06.132 INFO:teuthology.task.internal:Checking packages... 2024-04-25T20:02:06.158 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '646b8c7d293d8ec31a40c69f7b3237b19c1be24d' 2024-04-25T20:02:06.158 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T20:02:06.158 INFO:teuthology.packaging:ref: None 2024-04-25T20:02:06.158 INFO:teuthology.packaging:tag: None 2024-04-25T20:02:06.158 INFO:teuthology.packaging:branch: reef 2024-04-25T20:02:06.159 INFO:teuthology.packaging:sha1: 646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-25T20:02:06.159 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-04-25T20:02:06.401 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1258.g7d2137f0 2024-04-25T20:02:06.403 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T20:02:06.410 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T20:02:06.411 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T20:02:06.425 INFO:teuthology.task.internal:Saving configuration 2024-04-25T20:02:06.437 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T20:02:06.443 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T20:02:06.469 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi039.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-23_22:16:02-rbd-reef-distro-default-smithi/7670822', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 19:56:17.963570', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPx7DNlgqpi1ZwRCwkvSCxi2BlSMk8UY6pu4oQo7xnMWVcb1sBPhUbx41iRbpN+5U6ckuXt/05601eWN/K2B+Jw='} 2024-04-25T20:02:06.489 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi135.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-23_22:16:02-rbd-reef-distro-default-smithi/7670822', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 19:56:17.965429', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBONrHp0x4i8dM+RTeVczM1nKavKGfa39I1L1k+GaFeYF9LPF9lEqkUkFW7f1dKw8nW8rHjX+IdiF6GXTDpdianQ='} 2024-04-25T20:02:06.489 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T20:02:06.496 INFO:teuthology.task.internal:roles: ubuntu@smithi039.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-04-25T20:02:06.496 INFO:teuthology.task.internal:roles: ubuntu@smithi135.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-04-25T20:02:06.496 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T20:02:06.579 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff64d3833a0>, signals=[15]) 2024-04-25T20:02:06.580 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T20:02:06.589 INFO:teuthology.task.internal:Opening connections... 2024-04-25T20:02:06.589 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi039.front.sepia.ceph.com 2024-04-25T20:02:06.592 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T20:02:06.674 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi135.front.sepia.ceph.com 2024-04-25T20:02:06.675 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T20:02:06.749 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T20:02:06.756 DEBUG:teuthology.orchestra.run.smithi039:> uname -m 2024-04-25T20:02:06.775 INFO:teuthology.orchestra.run.smithi039.stdout:x86_64 2024-04-25T20:02:06.775 DEBUG:teuthology.orchestra.run.smithi039:> cat /etc/os-release 2024-04-25T20:02:06.833 INFO:teuthology.orchestra.run.smithi039.stdout:NAME="CentOS Stream" 2024-04-25T20:02:06.833 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION="9" 2024-04-25T20:02:06.833 INFO:teuthology.orchestra.run.smithi039.stdout:ID="centos" 2024-04-25T20:02:06.833 INFO:teuthology.orchestra.run.smithi039.stdout:ID_LIKE="rhel fedora" 2024-04-25T20:02:06.833 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_ID="9" 2024-04-25T20:02:06.833 INFO:teuthology.orchestra.run.smithi039.stdout:PLATFORM_ID="platform:el9" 2024-04-25T20:02:06.833 INFO:teuthology.orchestra.run.smithi039.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T20:02:06.833 INFO:teuthology.orchestra.run.smithi039.stdout:ANSI_COLOR="0;31" 2024-04-25T20:02:06.833 INFO:teuthology.orchestra.run.smithi039.stdout:LOGO="fedora-logo-icon" 2024-04-25T20:02:06.833 INFO:teuthology.orchestra.run.smithi039.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T20:02:06.833 INFO:teuthology.orchestra.run.smithi039.stdout:HOME_URL="https://centos.org/" 2024-04-25T20:02:06.834 INFO:teuthology.orchestra.run.smithi039.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T20:02:06.834 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T20:02:06.834 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T20:02:06.834 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2024-04-25T20:02:06.873 DEBUG:teuthology.orchestra.run.smithi135:> uname -m 2024-04-25T20:02:06.892 INFO:teuthology.orchestra.run.smithi135.stdout:x86_64 2024-04-25T20:02:06.893 DEBUG:teuthology.orchestra.run.smithi135:> cat /etc/os-release 2024-04-25T20:02:06.949 INFO:teuthology.orchestra.run.smithi135.stdout:NAME="CentOS Stream" 2024-04-25T20:02:06.949 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION="9" 2024-04-25T20:02:06.949 INFO:teuthology.orchestra.run.smithi135.stdout:ID="centos" 2024-04-25T20:02:06.950 INFO:teuthology.orchestra.run.smithi135.stdout:ID_LIKE="rhel fedora" 2024-04-25T20:02:06.950 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION_ID="9" 2024-04-25T20:02:06.950 INFO:teuthology.orchestra.run.smithi135.stdout:PLATFORM_ID="platform:el9" 2024-04-25T20:02:06.950 INFO:teuthology.orchestra.run.smithi135.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T20:02:06.950 INFO:teuthology.orchestra.run.smithi135.stdout:ANSI_COLOR="0;31" 2024-04-25T20:02:06.950 INFO:teuthology.orchestra.run.smithi135.stdout:LOGO="fedora-logo-icon" 2024-04-25T20:02:06.950 INFO:teuthology.orchestra.run.smithi135.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T20:02:06.950 INFO:teuthology.orchestra.run.smithi135.stdout:HOME_URL="https://centos.org/" 2024-04-25T20:02:06.950 INFO:teuthology.orchestra.run.smithi135.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T20:02:06.950 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T20:02:06.950 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T20:02:06.951 INFO:teuthology.lock.ops:Updating smithi135.front.sepia.ceph.com on lock server 2024-04-25T20:02:06.973 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T20:02:06.984 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T20:02:06.990 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T20:02:06.990 DEBUG:teuthology.orchestra.run.smithi039:> test '!' -e /home/ubuntu/cephtest 2024-04-25T20:02:06.993 DEBUG:teuthology.orchestra.run.smithi135:> test '!' -e /home/ubuntu/cephtest 2024-04-25T20:02:07.009 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T20:02:07.015 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T20:02:07.015 DEBUG:teuthology.orchestra.run.smithi039:> test -z $(ls -A /var/lib/ceph) 2024-04-25T20:02:07.050 DEBUG:teuthology.orchestra.run.smithi135:> test -z $(ls -A /var/lib/ceph) 2024-04-25T20:02:07.144 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T20:02:07.200 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T20:02:07.215 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-25T20:02:07.215 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-25T20:02:07.216 DEBUG:teuthology.orchestra.run.smithi039:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T20:02:07.216 DEBUG:teuthology.orchestra.run.smithi135:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T20:02:07.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:02:07.233 DEBUG:teuthology.orchestra.run.smithi039:> uname -r 2024-04-25T20:02:07.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:02:07.234 DEBUG:teuthology.orchestra.run.smithi135:> uname -r 2024-04-25T20:02:07.291 INFO:teuthology.orchestra.run.smithi039.stdout:5.14.0-437.el9.x86_64 2024-04-25T20:02:07.291 INFO:teuthology.task.kernel:Running kernel on smithi039: 5.14.0-437.el9.x86_64 2024-04-25T20:02:07.291 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum install -y kernel 2024-04-25T20:02:07.292 INFO:teuthology.orchestra.run.smithi135.stdout:5.14.0-437.el9.x86_64 2024-04-25T20:02:07.292 INFO:teuthology.task.kernel:Running kernel on smithi135: 5.14.0-437.el9.x86_64 2024-04-25T20:02:07.293 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum install -y kernel 2024-04-25T20:02:09.677 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-25T20:02:10.312 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - BaseOS 5.9 MB/s | 8.0 MB 00:01 2024-04-25T20:02:11.754 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-25T20:02:12.288 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-25T20:02:16.021 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-25T20:02:16.389 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-25T20:02:17.444 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - Extras packages 73 kB/s | 16 kB 00:00 2024-04-25T20:02:18.487 INFO:teuthology.orchestra.run.smithi039.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-04-25T20:02:21.320 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - Extras packages 4.5 kB/s | 16 kB 00:03 2024-04-25T20:02:22.497 INFO:teuthology.orchestra.run.smithi135.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-04-25T20:02:23.728 INFO:teuthology.orchestra.run.smithi039.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-04-25T20:02:25.718 INFO:teuthology.orchestra.run.smithi039.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T20:02:25.718 INFO:teuthology.orchestra.run.smithi039.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T20:02:25.790 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-04-25T20:02:25.791 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2024-04-25T20:02:25.791 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-04-25T20:02:25.877 DEBUG:teuthology.orchestra.run.smithi039:> echo no | sudo yum reinstall kernel || true 2024-04-25T20:02:26.401 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 08:02:23 PM UTC. 2024-04-25T20:02:26.523 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-04-25T20:02:26.541 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:02:26.542 INFO:teuthology.orchestra.run.smithi039.stdout: Package Architecture Version Repository Size 2024-04-25T20:02:26.542 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:02:26.542 INFO:teuthology.orchestra.run.smithi039.stdout:Reinstalling: 2024-04-25T20:02:26.542 INFO:teuthology.orchestra.run.smithi039.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T20:02:26.542 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:02:26.542 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-04-25T20:02:26.542 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:02:26.542 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:02:26.542 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 255 k 2024-04-25T20:02:26.542 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 0 2024-04-25T20:02:26.543 INFO:teuthology.orchestra.run.smithi039.stderr:Operation aborted. 2024-04-25T20:02:26.595 INFO:teuthology.orchestra.run.smithi039.stdout:Is this ok [y/N]: 2024-04-25T20:02:26.595 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum reinstall -y kernel || true 2024-04-25T20:02:27.110 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:00:04 ago on Thu 25 Apr 2024 08:02:23 PM UTC. 2024-04-25T20:02:27.231 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-04-25T20:02:27.231 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:02:27.231 INFO:teuthology.orchestra.run.smithi039.stdout: Package Architecture Version Repository Size 2024-04-25T20:02:27.231 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:02:27.231 INFO:teuthology.orchestra.run.smithi039.stdout:Reinstalling: 2024-04-25T20:02:27.231 INFO:teuthology.orchestra.run.smithi039.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T20:02:27.231 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:02:27.231 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-04-25T20:02:27.232 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:02:27.232 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:02:27.232 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 255 k 2024-04-25T20:02:27.232 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 0 2024-04-25T20:02:27.232 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-04-25T20:02:27.772 INFO:teuthology.orchestra.run.smithi135.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-04-25T20:02:29.687 INFO:teuthology.orchestra.run.smithi135.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T20:02:29.687 INFO:teuthology.orchestra.run.smithi135.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T20:02:29.776 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-25T20:02:29.777 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2024-04-25T20:02:29.777 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-25T20:02:29.897 DEBUG:teuthology.orchestra.run.smithi135:> echo no | sudo yum reinstall kernel || true 2024-04-25T20:02:30.438 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 08:02:27 PM UTC. 2024-04-25T20:02:30.556 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-25T20:02:30.574 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:02:30.575 INFO:teuthology.orchestra.run.smithi135.stdout: Package Architecture Version Repository Size 2024-04-25T20:02:30.575 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:02:30.575 INFO:teuthology.orchestra.run.smithi135.stdout:Reinstalling: 2024-04-25T20:02:30.575 INFO:teuthology.orchestra.run.smithi135.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T20:02:30.575 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:02:30.575 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-04-25T20:02:30.575 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:02:30.575 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:02:30.575 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 255 k 2024-04-25T20:02:30.575 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 0 2024-04-25T20:02:30.576 INFO:teuthology.orchestra.run.smithi135.stderr:Operation aborted. 2024-04-25T20:02:30.632 INFO:teuthology.orchestra.run.smithi135.stdout:Is this ok [y/N]: 2024-04-25T20:02:30.632 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum reinstall -y kernel || true 2024-04-25T20:02:31.127 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:00:04 ago on Thu 25 Apr 2024 08:02:27 PM UTC. 2024-04-25T20:02:31.243 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-25T20:02:31.244 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:02:31.244 INFO:teuthology.orchestra.run.smithi135.stdout: Package Architecture Version Repository Size 2024-04-25T20:02:31.244 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:02:31.244 INFO:teuthology.orchestra.run.smithi135.stdout:Reinstalling: 2024-04-25T20:02:31.244 INFO:teuthology.orchestra.run.smithi135.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T20:02:31.244 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:02:31.244 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-04-25T20:02:31.244 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:02:31.244 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:02:31.245 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 255 k 2024-04-25T20:02:31.245 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 0 2024-04-25T20:02:31.245 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-04-25T20:02:31.466 INFO:teuthology.orchestra.run.smithi135.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.7 MB/s | 255 kB 00:00 2024-04-25T20:02:31.467 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:02:31.467 INFO:teuthology.orchestra.run.smithi135.stdout:Total 1.1 MB/s | 255 kB 00:00 2024-04-25T20:02:31.481 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-04-25T20:02:31.665 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-04-25T20:02:31.665 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-04-25T20:02:31.670 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-04-25T20:02:31.670 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-04-25T20:02:31.800 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-04-25T20:02:31.884 INFO:teuthology.orchestra.run.smithi135.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T20:02:31.884 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T20:02:32.213 INFO:teuthology.orchestra.run.smithi039.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.8 MB/s | 255 kB 00:00 2024-04-25T20:02:32.214 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:02:32.214 INFO:teuthology.orchestra.run.smithi039.stdout:Total 51 kB/s | 255 kB 00:04 2024-04-25T20:02:32.231 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-04-25T20:02:32.416 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-04-25T20:02:32.416 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-04-25T20:02:32.421 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-04-25T20:02:32.422 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-04-25T20:02:32.554 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T20:02:32.554 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T20:02:32.567 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-04-25T20:02:32.639 INFO:teuthology.orchestra.run.smithi039.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T20:02:32.640 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T20:02:33.269 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T20:02:33.269 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T20:02:33.277 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T20:02:33.278 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:02:33.278 INFO:teuthology.orchestra.run.smithi135.stdout:Reinstalled: 2024-04-25T20:02:33.278 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T20:02:33.278 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:02:33.278 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-25T20:02:33.447 DEBUG:teuthology.orchestra.run.smithi135:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T20:02:33.493 INFO:teuthology.orchestra.run.smithi135.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T20:02:33.493 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T20:02:33.493 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T20:02:33.493 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-25T20:02:33.494 DEBUG:teuthology.orchestra.run.smithi135:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T20:02:33.527 INFO:teuthology.orchestra.run.smithi135.stdout:ttyS1 2024-04-25T20:02:33.554 DEBUG:teuthology.parallel:result is None 2024-04-25T20:02:34.084 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T20:02:34.084 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:02:34.085 INFO:teuthology.orchestra.run.smithi039.stdout:Reinstalled: 2024-04-25T20:02:34.085 INFO:teuthology.orchestra.run.smithi039.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T20:02:34.085 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:02:34.085 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-04-25T20:02:34.251 DEBUG:teuthology.orchestra.run.smithi039:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T20:02:34.299 INFO:teuthology.orchestra.run.smithi039.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T20:02:34.299 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T20:02:34.299 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T20:02:34.299 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-25T20:02:34.300 DEBUG:teuthology.orchestra.run.smithi039:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T20:02:34.328 INFO:teuthology.orchestra.run.smithi039.stdout:ttyS1 2024-04-25T20:02:34.348 DEBUG:teuthology.parallel:result is None 2024-04-25T20:02:34.348 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T20:02:34.356 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T20:02:34.356 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T20:02:34.391 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T20:02:34.408 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T20:02:34.415 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T20:02:34.425 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T20:02:34.425 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T20:02:34.446 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T20:02:34.474 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T20:02:34.482 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T20:02:34.482 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T20:02:34.506 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T20:02:34.547 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T20:02:34.557 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T20:02:34.680 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T20:02:34.689 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T20:02:34.691 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T20:02:34.699 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T20:02:34.699 DEBUG:teuthology.orchestra.run.smithi039:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T20:02:34.701 DEBUG:teuthology.orchestra.run.smithi135:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T20:02:34.730 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T20:02:34.739 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T20:02:34.739 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T20:02:34.766 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T20:02:34.786 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T20:02:34.862 DEBUG:teuthology.orchestra.run.smithi039:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T20:02:34.911 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T20:02:34.989 DEBUG:teuthology.orchestra.run.smithi039:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T20:02:35.032 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-04-25T20:02:35.033 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T20:02:35.097 DEBUG:teuthology.orchestra.run.smithi135:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T20:02:35.141 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T20:02:35.197 DEBUG:teuthology.orchestra.run.smithi135:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T20:02:35.282 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T20:02:35.323 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-25T20:02:35.324 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T20:02:35.390 DEBUG:teuthology.orchestra.run.smithi039:> sudo service rsyslog restart 2024-04-25T20:02:35.392 DEBUG:teuthology.orchestra.run.smithi135:> sudo service rsyslog restart 2024-04-25T20:02:35.426 INFO:teuthology.orchestra.run.smithi039.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T20:02:35.471 INFO:teuthology.orchestra.run.smithi135.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T20:02:35.853 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T20:02:35.863 INFO:teuthology.task.internal:Starting timer... 2024-04-25T20:02:35.863 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T20:02:35.883 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T20:02:35.892 DEBUG:teuthology.orchestra.run.smithi039:> sudo service auditd rotate 2024-04-25T20:02:35.942 INFO:teuthology.orchestra.run.smithi039.stdout:Rotating logs: 2024-04-25T20:02:35.944 DEBUG:teuthology.orchestra.run.smithi135:> sudo service auditd rotate 2024-04-25T20:02:36.023 INFO:teuthology.orchestra.run.smithi135.stdout:Rotating logs: 2024-04-25T20:02:36.025 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T20:02:36.025 DEBUG:teuthology.orchestra.run.smithi039:> /usr/sbin/getenforce 2024-04-25T20:02:36.055 INFO:teuthology.orchestra.run.smithi039.stdout:Permissive 2024-04-25T20:02:36.055 DEBUG:teuthology.orchestra.run.smithi135:> /usr/sbin/getenforce 2024-04-25T20:02:36.076 INFO:teuthology.orchestra.run.smithi135.stdout:Permissive 2024-04-25T20:02:36.076 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi039.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi135.front.sepia.ceph.com': 'permissive'} 2024-04-25T20:02:36.076 DEBUG:teuthology.orchestra.run.smithi039:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T20:02:36.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:02:36.121 DEBUG:teuthology.orchestra.run.smithi135:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T20:02:36.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:02:36.147 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T20:02:36.147 DEBUG:teuthology.orchestra.run.smithi039:> sudo /usr/sbin/setenforce permissive 2024-04-25T20:02:36.187 DEBUG:teuthology.orchestra.run.smithi135:> sudo /usr/sbin/setenforce permissive 2024-04-25T20:02:36.212 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T20:02:36.222 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T20:02:36.239 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T20:02:36.240 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi039.front.sepia.ceph.com,smithi135.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T20:06:09.221 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi039.front.sepia.ceph.com'), Remote(name='ubuntu@smithi135.front.sepia.ceph.com')] 2024-04-25T20:06:09.224 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-04-25T20:06:09.225 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T20:06:09.306 DEBUG:teuthology.orchestra.run.smithi039:> true 2024-04-25T20:06:09.346 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-04-25T20:06:09.346 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-04-25T20:06:09.347 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T20:06:09.426 DEBUG:teuthology.orchestra.run.smithi135:> true 2024-04-25T20:06:09.508 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-04-25T20:06:09.508 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T20:06:09.519 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T20:06:09.519 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T20:06:09.519 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T20:06:09.522 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T20:06:09.522 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T20:06:09.547 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T20:06:09.561 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T20:06:09.576 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T20:06:09.584 INFO:teuthology.orchestra.run.smithi039.stderr:sudo: ntpd: command not found 2024-04-25T20:06:09.589 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T20:06:09.594 INFO:teuthology.orchestra.run.smithi039.stdout:506 Cannot talk to daemon 2024-04-25T20:06:09.608 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T20:06:09.611 INFO:teuthology.orchestra.run.smithi135.stderr:sudo: ntpd: command not found 2024-04-25T20:06:09.621 INFO:teuthology.orchestra.run.smithi135.stdout:506 Cannot talk to daemon 2024-04-25T20:06:09.621 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T20:06:09.634 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T20:06:09.646 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T20:06:09.676 INFO:teuthology.orchestra.run.smithi039.stderr:bash: line 1: ntpq: command not found 2024-04-25T20:06:09.679 INFO:teuthology.orchestra.run.smithi039.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T20:06:09.679 INFO:teuthology.orchestra.run.smithi039.stdout:=============================================================================== 2024-04-25T20:06:09.679 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:06:09.679 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:06:09.679 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:06:09.680 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:06:09.702 INFO:teuthology.orchestra.run.smithi135.stderr:bash: line 1: ntpq: command not found 2024-04-25T20:06:09.705 INFO:teuthology.orchestra.run.smithi135.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T20:06:09.705 INFO:teuthology.orchestra.run.smithi135.stdout:=============================================================================== 2024-04-25T20:06:09.705 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:06:09.705 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:06:09.706 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:06:09.706 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:06:09.707 INFO:teuthology.run_tasks:Running task install... 2024-04-25T20:06:09.715 DEBUG:teuthology.task.install:project ceph 2024-04-25T20:06:09.716 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '646b8c7d293d8ec31a40c69f7b3237b19c1be24d'}} 2024-04-25T20:06:09.716 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '646b8c7d293d8ec31a40c69f7b3237b19c1be24d'} 2024-04-25T20:06:09.716 INFO:teuthology.task.install:Using flavor: default 2024-04-25T20:06:09.721 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T20:06:09.721 INFO:teuthology.task.install:extra packages: [] 2024-04-25T20:06:09.722 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '646b8c7d293d8ec31a40c69f7b3237b19c1be24d', 'tag': None, 'wait_for_package': False} 2024-04-25T20:06:09.722 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-25T20:06:09.724 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '646b8c7d293d8ec31a40c69f7b3237b19c1be24d', 'tag': None, 'wait_for_package': False} 2024-04-25T20:06:09.725 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-25T20:06:09.869 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/centos/9/flavors/default/ 2024-04-25T20:06:09.870 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1237.g646b8c7d 2024-04-25T20:06:09.875 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/centos/9/flavors/default/ 2024-04-25T20:06:09.875 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1237.g646b8c7d 2024-04-25T20:06:10.046 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T20:06:10.047 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-04-25T20:06:10.047 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T20:06:10.052 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T20:06:10.052 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-25T20:06:10.052 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T20:06:10.073 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-25T20:06:10.073 DEBUG:teuthology.orchestra.run.smithi039:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T20:06:10.076 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-25T20:06:10.076 DEBUG:teuthology.orchestra.run.smithi135:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T20:06:10.139 DEBUG:teuthology.orchestra.run.smithi039:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T20:06:10.141 DEBUG:teuthology.orchestra.run.smithi135:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T20:06:10.214 DEBUG:teuthology.orchestra.run.smithi039:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T20:06:10.215 DEBUG:teuthology.orchestra.run.smithi135:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T20:06:10.277 INFO:teuthology.orchestra.run.smithi039.stdout:check_obsoletes = 1 2024-04-25T20:06:10.278 INFO:teuthology.orchestra.run.smithi135.stdout:check_obsoletes = 1 2024-04-25T20:06:10.279 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum clean all 2024-04-25T20:06:10.280 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum clean all 2024-04-25T20:06:10.508 INFO:teuthology.orchestra.run.smithi135.stdout:48 files removed 2024-04-25T20:06:10.512 INFO:teuthology.orchestra.run.smithi039.stdout:48 files removed 2024-04-25T20:06:10.535 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-radosgw 2024-04-25T20:06:10.539 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-radosgw 2024-04-25T20:06:11.184 INFO:teuthology.orchestra.run.smithi135.stdout:ceph packages for x86_64 196 kB/s | 77 kB 00:00 2024-04-25T20:06:11.233 INFO:teuthology.orchestra.run.smithi039.stdout:ceph packages for x86_64 182 kB/s | 77 kB 00:00 2024-04-25T20:06:11.510 INFO:teuthology.orchestra.run.smithi135.stdout:ceph noarch packages 40 kB/s | 11 kB 00:00 2024-04-25T20:06:11.609 INFO:teuthology.orchestra.run.smithi039.stdout:ceph noarch packages 34 kB/s | 11 kB 00:00 2024-04-25T20:06:11.942 INFO:teuthology.orchestra.run.smithi039.stdout:ceph source packages 6.3 kB/s | 1.9 kB 00:00 2024-04-25T20:06:11.994 INFO:teuthology.orchestra.run.smithi135.stdout:ceph source packages 4.2 kB/s | 1.9 kB 00:00 2024-04-25T20:06:12.562 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-25T20:06:12.755 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-25T20:06:14.658 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-25T20:06:14.788 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-04-25T20:06:19.009 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - CRB 8.2 MB/s | 6.2 MB 00:00 2024-04-25T20:06:19.264 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - CRB 8.6 MB/s | 6.2 MB 00:00 2024-04-25T20:06:20.690 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 16 kB 00:00 2024-04-25T20:06:20.820 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 16 kB 00:00 2024-04-25T20:06:22.095 INFO:teuthology.orchestra.run.smithi135.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-25T20:06:22.324 INFO:teuthology.orchestra.run.smithi039.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-25T20:06:27.109 INFO:teuthology.orchestra.run.smithi135.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2024-04-25T20:06:27.555 INFO:teuthology.orchestra.run.smithi039.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-04-25T20:06:29.278 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-25T20:06:29.280 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:06:29.280 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-04-25T20:06:29.280 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:06:29.280 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-04-25T20:06:29.280 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-radosgw x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 7.7 M 2024-04-25T20:06:29.280 INFO:teuthology.orchestra.run.smithi135.stdout:Upgrading: 2024-04-25T20:06:29.281 INFO:teuthology.orchestra.run.smithi135.stdout: librados2 x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 3.3 M 2024-04-25T20:06:29.281 INFO:teuthology.orchestra.run.smithi135.stdout: librbd1 x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 3.0 M 2024-04-25T20:06:29.281 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-04-25T20:06:29.281 INFO:teuthology.orchestra.run.smithi135.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T20:06:29.281 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-base x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 5.1 M 2024-04-25T20:06:29.281 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-common x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 18 M 2024-04-25T20:06:29.281 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-selinux x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 25 k 2024-04-25T20:06:29.281 INFO:teuthology.orchestra.run.smithi135.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-04-25T20:06:29.281 INFO:teuthology.orchestra.run.smithi135.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T20:06:29.281 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T20:06:29.281 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T20:06:29.281 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs2 x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 709 k 2024-04-25T20:06:29.281 INFO:teuthology.orchestra.run.smithi135.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T20:06:29.281 INFO:teuthology.orchestra.run.smithi135.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T20:06:29.282 INFO:teuthology.orchestra.run.smithi135.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T20:06:29.282 INFO:teuthology.orchestra.run.smithi135.stdout: libradosstriper1 x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 474 k 2024-04-25T20:06:29.282 INFO:teuthology.orchestra.run.smithi135.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T20:06:29.282 INFO:teuthology.orchestra.run.smithi135.stdout: librgw2 x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 4.5 M 2024-04-25T20:06:29.282 INFO:teuthology.orchestra.run.smithi135.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T20:06:29.282 INFO:teuthology.orchestra.run.smithi135.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T20:06:29.282 INFO:teuthology.orchestra.run.smithi135.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T20:06:29.282 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-argparse x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 45 k 2024-04-25T20:06:29.282 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-common x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 129 k 2024-04-25T20:06:29.282 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cephfs x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 161 k 2024-04-25T20:06:29.282 INFO:teuthology.orchestra.run.smithi135.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T20:06:29.283 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rados x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 320 k 2024-04-25T20:06:29.283 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rbd x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 298 k 2024-04-25T20:06:29.283 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rgw x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 99 k 2024-04-25T20:06:29.283 INFO:teuthology.orchestra.run.smithi135.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T20:06:29.283 INFO:teuthology.orchestra.run.smithi135.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T20:06:29.283 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:06:29.283 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-04-25T20:06:29.283 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:06:29.283 INFO:teuthology.orchestra.run.smithi135.stdout:Install 28 Packages 2024-04-25T20:06:29.283 INFO:teuthology.orchestra.run.smithi135.stdout:Upgrade 2 Packages 2024-04-25T20:06:29.283 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:06:29.284 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 53 M 2024-04-25T20:06:29.284 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-04-25T20:06:29.767 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-04-25T20:06:29.769 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:06:29.769 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-04-25T20:06:29.769 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:06:29.769 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-04-25T20:06:29.769 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-radosgw x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 7.7 M 2024-04-25T20:06:29.770 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrading: 2024-04-25T20:06:29.770 INFO:teuthology.orchestra.run.smithi039.stdout: librados2 x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 3.3 M 2024-04-25T20:06:29.770 INFO:teuthology.orchestra.run.smithi039.stdout: librbd1 x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 3.0 M 2024-04-25T20:06:29.770 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2024-04-25T20:06:29.770 INFO:teuthology.orchestra.run.smithi039.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T20:06:29.770 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-base x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 5.1 M 2024-04-25T20:06:29.770 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-common x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 18 M 2024-04-25T20:06:29.770 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-selinux x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 25 k 2024-04-25T20:06:29.770 INFO:teuthology.orchestra.run.smithi039.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-04-25T20:06:29.770 INFO:teuthology.orchestra.run.smithi039.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T20:06:29.770 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T20:06:29.770 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T20:06:29.770 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs2 x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 709 k 2024-04-25T20:06:29.771 INFO:teuthology.orchestra.run.smithi039.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T20:06:29.771 INFO:teuthology.orchestra.run.smithi039.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T20:06:29.771 INFO:teuthology.orchestra.run.smithi039.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T20:06:29.771 INFO:teuthology.orchestra.run.smithi039.stdout: libradosstriper1 x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 474 k 2024-04-25T20:06:29.771 INFO:teuthology.orchestra.run.smithi039.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T20:06:29.771 INFO:teuthology.orchestra.run.smithi039.stdout: librgw2 x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 4.5 M 2024-04-25T20:06:29.771 INFO:teuthology.orchestra.run.smithi039.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T20:06:29.771 INFO:teuthology.orchestra.run.smithi039.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T20:06:29.771 INFO:teuthology.orchestra.run.smithi039.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T20:06:29.771 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-argparse x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 45 k 2024-04-25T20:06:29.771 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-common x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 129 k 2024-04-25T20:06:29.771 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cephfs x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 161 k 2024-04-25T20:06:29.771 INFO:teuthology.orchestra.run.smithi039.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T20:06:29.771 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rados x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 320 k 2024-04-25T20:06:29.772 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rbd x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 298 k 2024-04-25T20:06:29.772 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rgw x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 99 k 2024-04-25T20:06:29.772 INFO:teuthology.orchestra.run.smithi039.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T20:06:29.772 INFO:teuthology.orchestra.run.smithi039.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T20:06:29.772 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:06:29.772 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-04-25T20:06:29.772 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:06:29.772 INFO:teuthology.orchestra.run.smithi039.stdout:Install 28 Packages 2024-04-25T20:06:29.772 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrade 2 Packages 2024-04-25T20:06:29.772 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:06:29.773 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 53 M 2024-04-25T20:06:29.773 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-04-25T20:06:30.627 INFO:teuthology.orchestra.run.smithi135.stdout:(1/30): ceph-radosgw-18.2.2-1237.g646b8c7d.el8. 11 MB/s | 7.7 MB 00:00 2024-04-25T20:06:30.677 INFO:teuthology.orchestra.run.smithi135.stdout:(2/30): ceph-selinux-18.2.2-1237.g646b8c7d.el8. 505 kB/s | 25 kB 00:00 2024-04-25T20:06:30.778 INFO:teuthology.orchestra.run.smithi135.stdout:(3/30): libcephfs2-18.2.2-1237.g646b8c7d.el8.x8 6.9 MB/s | 709 kB 00:00 2024-04-25T20:06:30.837 INFO:teuthology.orchestra.run.smithi135.stdout:(4/30): libradosstriper1-18.2.2-1237.g646b8c7d. 8.0 MB/s | 474 kB 00:00 2024-04-25T20:06:31.153 INFO:teuthology.orchestra.run.smithi135.stdout:(5/30): librgw2-18.2.2-1237.g646b8c7d.el8.x86_6 14 MB/s | 4.5 MB 00:00 2024-04-25T20:06:31.196 INFO:teuthology.orchestra.run.smithi135.stdout:(6/30): python3-ceph-argparse-18.2.2-1237.g646b 1.1 MB/s | 45 kB 00:00 2024-04-25T20:06:31.198 INFO:teuthology.orchestra.run.smithi039.stdout:(1/30): ceph-base-18.2.2-1237.g646b8c7d.el8.x86 4.3 MB/s | 5.1 MB 00:01 2024-04-25T20:06:31.248 INFO:teuthology.orchestra.run.smithi039.stdout:(2/30): ceph-selinux-18.2.2-1237.g646b8c7d.el8. 507 kB/s | 25 kB 00:00 2024-04-25T20:06:31.262 INFO:teuthology.orchestra.run.smithi135.stdout:(7/30): python3-ceph-common-18.2.2-1237.g646b8c 1.9 MB/s | 129 kB 00:00 2024-04-25T20:06:31.313 INFO:teuthology.orchestra.run.smithi135.stdout:(8/30): python3-cephfs-18.2.2-1237.g646b8c7d.el 3.1 MB/s | 161 kB 00:00 2024-04-25T20:06:31.365 INFO:teuthology.orchestra.run.smithi039.stdout:(3/30): libcephfs2-18.2.2-1237.g646b8c7d.el8.x8 5.9 MB/s | 709 kB 00:00 2024-04-25T20:06:31.371 INFO:teuthology.orchestra.run.smithi135.stdout:(9/30): python3-rados-18.2.2-1237.g646b8c7d.el8 5.4 MB/s | 320 kB 00:00 2024-04-25T20:06:31.438 INFO:teuthology.orchestra.run.smithi135.stdout:(10/30): python3-rbd-18.2.2-1237.g646b8c7d.el8. 4.4 MB/s | 298 kB 00:00 2024-04-25T20:06:31.473 INFO:teuthology.orchestra.run.smithi039.stdout:(4/30): libradosstriper1-18.2.2-1237.g646b8c7d. 4.3 MB/s | 474 kB 00:00 2024-04-25T20:06:31.489 INFO:teuthology.orchestra.run.smithi135.stdout:(11/30): python3-rgw-18.2.2-1237.g646b8c7d.el8. 1.9 MB/s | 99 kB 00:00 2024-04-25T20:06:31.614 INFO:teuthology.orchestra.run.smithi135.stdout:(12/30): boost-program-options-1.75.0-8.el9.x86 855 kB/s | 107 kB 00:00 2024-04-25T20:06:31.681 INFO:teuthology.orchestra.run.smithi135.stdout:(13/30): ceph-base-18.2.2-1237.g646b8c7d.el8.x8 2.9 MB/s | 5.1 MB 00:01 2024-04-25T20:06:31.706 INFO:teuthology.orchestra.run.smithi135.stdout:(14/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-04-25T20:06:31.732 INFO:teuthology.orchestra.run.smithi135.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-04-25T20:06:31.791 INFO:teuthology.orchestra.run.smithi135.stdout:(16/30): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2024-04-25T20:06:31.816 INFO:teuthology.orchestra.run.smithi135.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-04-25T20:06:31.841 INFO:teuthology.orchestra.run.smithi135.stdout:(18/30): python3-prettytable-0.7.2-27.el9.noarc 846 kB/s | 42 kB 00:00 2024-04-25T20:06:31.957 INFO:teuthology.orchestra.run.smithi039.stdout:(5/30): librgw2-18.2.2-1237.g646b8c7d.el8.x86_6 9.2 MB/s | 4.5 MB 00:00 2024-04-25T20:06:31.999 INFO:teuthology.orchestra.run.smithi039.stdout:(6/30): python3-ceph-argparse-18.2.2-1237.g646b 1.1 MB/s | 45 kB 00:00 2024-04-25T20:06:32.058 INFO:teuthology.orchestra.run.smithi039.stdout:(7/30): python3-ceph-common-18.2.2-1237.g646b8c 2.2 MB/s | 129 kB 00:00 2024-04-25T20:06:32.066 INFO:teuthology.orchestra.run.smithi135.stdout:(19/30): fmt-8.1.1-5.el9.x86_64.rpm 442 kB/s | 111 kB 00:00 2024-04-25T20:06:32.125 INFO:teuthology.orchestra.run.smithi039.stdout:(8/30): python3-cephfs-18.2.2-1237.g646b8c7d.el 2.4 MB/s | 161 kB 00:00 2024-04-25T20:06:32.150 INFO:teuthology.orchestra.run.smithi135.stdout:(20/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.0 MB/s | 309 kB 00:00 2024-04-25T20:06:32.209 INFO:teuthology.orchestra.run.smithi135.stdout:(21/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 446 kB/s | 26 kB 00:00 2024-04-25T20:06:32.241 INFO:teuthology.orchestra.run.smithi039.stdout:(9/30): ceph-radosgw-18.2.2-1237.g646b8c7d.el8. 3.5 MB/s | 7.7 MB 00:02 2024-04-25T20:06:32.267 INFO:teuthology.orchestra.run.smithi039.stdout:(10/30): python3-rados-18.2.2-1237.g646b8c7d.el 2.2 MB/s | 320 kB 00:00 2024-04-25T20:06:32.276 INFO:teuthology.orchestra.run.smithi135.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 729 kB/s | 49 kB 00:00 2024-04-25T20:06:32.309 INFO:teuthology.orchestra.run.smithi039.stdout:(11/30): python3-rgw-18.2.2-1237.g646b8c7d.el8. 2.3 MB/s | 99 kB 00:00 2024-04-25T20:06:32.334 INFO:teuthology.orchestra.run.smithi039.stdout:(12/30): python3-rbd-18.2.2-1237.g646b8c7d.el8. 3.2 MB/s | 298 kB 00:00 2024-04-25T20:06:32.343 INFO:teuthology.orchestra.run.smithi135.stdout:(23/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-25T20:06:32.418 INFO:teuthology.orchestra.run.smithi135.stdout:(24/30): libarrow-9.0.0-10.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-04-25T20:06:32.468 INFO:teuthology.orchestra.run.smithi039.stdout:(13/30): boost-program-options-1.75.0-8.el9.x86 674 kB/s | 107 kB 00:00 2024-04-25T20:06:32.477 INFO:teuthology.orchestra.run.smithi135.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-04-25T20:06:32.493 INFO:teuthology.orchestra.run.smithi039.stdout:(14/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-04-25T20:06:32.512 INFO:teuthology.orchestra.run.smithi135.stdout:(26/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.9 MB/s | 838 kB 00:00 2024-04-25T20:06:32.518 INFO:teuthology.orchestra.run.smithi039.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-04-25T20:06:32.562 INFO:teuthology.orchestra.run.smithi135.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-04-25T20:06:32.576 INFO:teuthology.orchestra.run.smithi039.stdout:(16/30): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 662 kB 00:00 2024-04-25T20:06:32.602 INFO:teuthology.orchestra.run.smithi039.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-04-25T20:06:32.628 INFO:teuthology.orchestra.run.smithi039.stdout:(18/30): python3-prettytable-0.7.2-27.el9.noarc 838 kB/s | 42 kB 00:00 2024-04-25T20:06:32.846 INFO:teuthology.orchestra.run.smithi135.stdout:(28/30): librbd1-18.2.2-1237.g646b8c7d.el8.x86_ 11 MB/s | 3.0 MB 00:00 2024-04-25T20:06:32.945 INFO:teuthology.orchestra.run.smithi039.stdout:(19/30): fmt-8.1.1-5.el9.x86_64.rpm 324 kB/s | 111 kB 00:00 2024-04-25T20:06:32.970 INFO:teuthology.orchestra.run.smithi039.stdout:(20/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 904 kB/s | 309 kB 00:00 2024-04-25T20:06:33.029 INFO:teuthology.orchestra.run.smithi039.stdout:(21/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 445 kB/s | 26 kB 00:00 2024-04-25T20:06:33.087 INFO:teuthology.orchestra.run.smithi039.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 835 kB/s | 49 kB 00:00 2024-04-25T20:06:33.146 INFO:teuthology.orchestra.run.smithi039.stdout:(23/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-25T20:06:33.330 INFO:teuthology.orchestra.run.smithi039.stdout:(24/30): ceph-common-18.2.2-1237.g646b8c7d.el8. 5.5 MB/s | 18 MB 00:03 2024-04-25T20:06:33.346 INFO:teuthology.orchestra.run.smithi135.stdout:(29/30): librados2-18.2.2-1237.g646b8c7d.el8.x8 4.0 MB/s | 3.3 MB 00:00 2024-04-25T20:06:33.363 INFO:teuthology.orchestra.run.smithi039.stdout:(25/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.8 MB/s | 838 kB 00:00 2024-04-25T20:06:33.430 INFO:teuthology.orchestra.run.smithi039.stdout:(26/30): libarrow-9.0.0-10.el9.x86_64.rpm 9.1 MB/s | 4.4 MB 00:00 2024-04-25T20:06:33.497 INFO:teuthology.orchestra.run.smithi039.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-25T20:06:33.564 INFO:teuthology.orchestra.run.smithi039.stdout:(28/30): re2-20211101-3.el9.x86_64.rpm 820 kB/s | 192 kB 00:00 2024-04-25T20:06:33.755 INFO:teuthology.orchestra.run.smithi039.stdout:(29/30): librbd1-18.2.2-1237.g646b8c7d.el8.x86_ 12 MB/s | 3.0 MB 00:00 2024-04-25T20:06:33.906 INFO:teuthology.orchestra.run.smithi039.stdout:(30/30): librados2-18.2.2-1237.g646b8c7d.el8.x8 6.9 MB/s | 3.3 MB 00:00 2024-04-25T20:06:33.911 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:06:33.911 INFO:teuthology.orchestra.run.smithi039.stdout:Total 13 MB/s | 53 MB 00:04 2024-04-25T20:06:33.986 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-04-25T20:06:34.027 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-04-25T20:06:34.027 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-04-25T20:06:34.472 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-04-25T20:06:34.472 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-04-25T20:06:35.858 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-04-25T20:06:35.943 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-04-25T20:06:35.980 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-04-25T20:06:36.030 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-04-25T20:06:36.358 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-04-25T20:06:36.478 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : librados2-2:18.2.2-1237.g646b8c7d.el8.x86_64 5/32 2024-04-25T20:06:36.555 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librados2-2:18.2.2-1237.g646b8c7d.el8.x86_64 5/32 2024-04-25T20:06:36.641 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rados-2:18.2.2-1237.g646b8c7d.el8.x86_64 6/32 2024-04-25T20:06:36.738 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephfs2-2:18.2.2-1237.g646b8c7d.el8.x86_64 7/32 2024-04-25T20:06:36.801 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libcephfs2-2:18.2.2-1237.g646b8c7d.el8.x86_64 7/32 2024-04-25T20:06:36.830 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-04-25T20:06:36.874 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ceph-argparse-2:18.2.2-1237.g646b8c7d.el8. 9/32 2024-04-25T20:06:36.941 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cephfs-2:18.2.2-1237.g646b8c7d.el8.x86_64 10/32 2024-04-25T20:06:36.965 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libradosstriper1-2:18.2.2-1237.g646b8c7d.el8.x86_6 11/32 2024-04-25T20:06:37.031 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1237.g646b8c7d.el8.x86_6 11/32 2024-04-25T20:06:37.046 INFO:teuthology.orchestra.run.smithi135.stdout:(30/30): ceph-common-18.2.2-1237.g646b8c7d.el8. 2.6 MB/s | 18 MB 00:07 2024-04-25T20:06:37.050 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:06:37.050 INFO:teuthology.orchestra.run.smithi135.stdout:Total 6.8 MB/s | 53 MB 00:07 2024-04-25T20:06:37.122 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-04-25T20:06:37.149 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-04-25T20:06:37.158 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-04-25T20:06:37.158 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-04-25T20:06:37.181 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-04-25T20:06:37.221 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-04-25T20:06:37.255 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-04-25T20:06:37.282 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-04-25T20:06:37.318 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-04-25T20:06:37.350 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-04-25T20:06:37.582 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-04-25T20:06:37.582 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-04-25T20:06:37.666 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-04-25T20:06:37.704 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : librbd1-2:18.2.2-1237.g646b8c7d.el8.x86_64 20/32 2024-04-25T20:06:37.779 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librbd1-2:18.2.2-1237.g646b8c7d.el8.x86_64 20/32 2024-04-25T20:06:37.809 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rbd-2:18.2.2-1237.g646b8c7d.el8.x86_64 21/32 2024-04-25T20:06:37.846 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-04-25T20:06:38.269 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-04-25T20:06:38.299 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librgw2-2:18.2.2-1237.g646b8c7d.el8.x86_64 24/32 2024-04-25T20:06:38.370 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librgw2-2:18.2.2-1237.g646b8c7d.el8.x86_64 24/32 2024-04-25T20:06:38.450 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rgw-2:18.2.2-1237.g646b8c7d.el8.x86_64 25/32 2024-04-25T20:06:38.476 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ceph-common-2:18.2.2-1237.g646b8c7d.el8.x8 26/32 2024-04-25T20:06:38.972 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-04-25T20:06:39.063 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-04-25T20:06:39.093 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-04-25T20:06:39.142 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-04-25T20:06:39.455 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-04-25T20:06:39.568 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : librados2-2:18.2.2-1237.g646b8c7d.el8.x86_64 5/32 2024-04-25T20:06:39.651 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados2-2:18.2.2-1237.g646b8c7d.el8.x86_64 5/32 2024-04-25T20:06:39.738 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rados-2:18.2.2-1237.g646b8c7d.el8.x86_64 6/32 2024-04-25T20:06:39.771 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephfs2-2:18.2.2-1237.g646b8c7d.el8.x86_64 7/32 2024-04-25T20:06:39.826 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephfs2-2:18.2.2-1237.g646b8c7d.el8.x86_64 7/32 2024-04-25T20:06:39.854 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-04-25T20:06:39.898 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ceph-argparse-2:18.2.2-1237.g646b8c7d.el8. 9/32 2024-04-25T20:06:39.964 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cephfs-2:18.2.2-1237.g646b8c7d.el8.x86_64 10/32 2024-04-25T20:06:39.991 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libradosstriper1-2:18.2.2-1237.g646b8c7d.el8.x86_6 11/32 2024-04-25T20:06:40.056 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1237.g646b8c7d.el8.x86_6 11/32 2024-04-25T20:06:40.163 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-04-25T20:06:40.189 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-04-25T20:06:40.230 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-04-25T20:06:40.263 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-04-25T20:06:40.291 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-04-25T20:06:40.327 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-04-25T20:06:40.360 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-04-25T20:06:40.661 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-04-25T20:06:40.674 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-common-2:18.2.2-1237.g646b8c7d.el8.x86_64 27/32 2024-04-25T20:06:40.698 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : librbd1-2:18.2.2-1237.g646b8c7d.el8.x86_64 20/32 2024-04-25T20:06:40.714 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-common-2:18.2.2-1237.g646b8c7d.el8.x86_64 27/32 2024-04-25T20:06:40.789 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librbd1-2:18.2.2-1237.g646b8c7d.el8.x86_64 20/32 2024-04-25T20:06:40.819 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rbd-2:18.2.2-1237.g646b8c7d.el8.x86_64 21/32 2024-04-25T20:06:40.880 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-04-25T20:06:41.265 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-common-2:18.2.2-1237.g646b8c7d.el8.x86_64 27/32 2024-04-25T20:06:41.282 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-04-25T20:06:41.300 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-base-2:18.2.2-1237.g646b8c7d.el8.x86_64 28/32 2024-04-25T20:06:41.309 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librgw2-2:18.2.2-1237.g646b8c7d.el8.x86_64 24/32 2024-04-25T20:06:41.381 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librgw2-2:18.2.2-1237.g646b8c7d.el8.x86_64 24/32 2024-04-25T20:06:41.392 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-base-2:18.2.2-1237.g646b8c7d.el8.x86_64 28/32 2024-04-25T20:06:41.393 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T20:06:41.393 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T20:06:41.393 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:06:41.427 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-selinux-2:18.2.2-1237.g646b8c7d.el8.x86_64 29/32 2024-04-25T20:06:41.448 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rgw-2:18.2.2-1237.g646b8c7d.el8.x86_64 25/32 2024-04-25T20:06:41.487 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ceph-common-2:18.2.2-1237.g646b8c7d.el8.x8 26/32 2024-04-25T20:06:43.568 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-common-2:18.2.2-1237.g646b8c7d.el8.x86_64 27/32 2024-04-25T20:06:43.602 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-common-2:18.2.2-1237.g646b8c7d.el8.x86_64 27/32 2024-04-25T20:06:44.125 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-common-2:18.2.2-1237.g646b8c7d.el8.x86_64 27/32 2024-04-25T20:06:44.173 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-base-2:18.2.2-1237.g646b8c7d.el8.x86_64 28/32 2024-04-25T20:06:44.260 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-base-2:18.2.2-1237.g646b8c7d.el8.x86_64 28/32 2024-04-25T20:06:44.261 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T20:06:44.261 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T20:06:44.261 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:06:44.330 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-selinux-2:18.2.2-1237.g646b8c7d.el8.x86_64 29/32 2024-04-25T20:06:51.481 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1237.g646b8c7d.el8.x86_64 29/32 2024-04-25T20:06:51.481 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /sys 2024-04-25T20:06:51.482 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /proc 2024-04-25T20:06:51.482 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /mnt 2024-04-25T20:06:51.482 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /var/tmp 2024-04-25T20:06:51.482 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /home 2024-04-25T20:06:51.482 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /root 2024-04-25T20:06:51.482 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /tmp 2024-04-25T20:06:51.482 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:06:52.259 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-radosgw-2:18.2.2-1237.g646b8c7d.el8.x86_64 30/32 2024-04-25T20:06:52.285 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1237.g646b8c7d.el8.x86_64 30/32 2024-04-25T20:06:52.285 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:06:52.285 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T20:06:52.285 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T20:06:52.285 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T20:06:52.285 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:06:52.287 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-25T20:06:52.370 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-25T20:06:52.370 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-25T20:06:53.553 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-25T20:06:53.553 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-base-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/32 2024-04-25T20:06:53.553 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-common-2:18.2.2-1237.g646b8c7d.el8.x86_64 2/32 2024-04-25T20:06:53.553 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-radosgw-2:18.2.2-1237.g646b8c7d.el8.x86_64 3/32 2024-04-25T20:06:53.553 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-selinux-2:18.2.2-1237.g646b8c7d.el8.x86_64 4/32 2024-04-25T20:06:53.553 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephfs2-2:18.2.2-1237.g646b8c7d.el8.x86_64 5/32 2024-04-25T20:06:53.553 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libradosstriper1-2:18.2.2-1237.g646b8c7d.el8.x86_6 6/32 2024-04-25T20:06:53.553 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librgw2-2:18.2.2-1237.g646b8c7d.el8.x86_64 7/32 2024-04-25T20:06:53.553 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1237.g646b8c7d.el8. 8/32 2024-04-25T20:06:53.553 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ceph-common-2:18.2.2-1237.g646b8c7d.el8.x8 9/32 2024-04-25T20:06:53.554 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cephfs-2:18.2.2-1237.g646b8c7d.el8.x86_64 10/32 2024-04-25T20:06:53.554 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rados-2:18.2.2-1237.g646b8c7d.el8.x86_64 11/32 2024-04-25T20:06:53.554 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rbd-2:18.2.2-1237.g646b8c7d.el8.x86_64 12/32 2024-04-25T20:06:53.554 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rgw-2:18.2.2-1237.g646b8c7d.el8.x86_64 13/32 2024-04-25T20:06:53.554 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-04-25T20:06:53.554 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-04-25T20:06:53.554 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-04-25T20:06:53.554 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-04-25T20:06:53.554 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-04-25T20:06:53.554 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-04-25T20:06:53.554 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-04-25T20:06:53.554 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-04-25T20:06:53.554 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-04-25T20:06:53.555 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-04-25T20:06:53.555 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-04-25T20:06:53.555 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-04-25T20:06:53.555 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-04-25T20:06:53.555 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-04-25T20:06:53.555 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-04-25T20:06:53.555 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados2-2:18.2.2-1237.g646b8c7d.el8.x86_64 29/32 2024-04-25T20:06:53.555 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-04-25T20:06:53.555 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librbd1-2:18.2.2-1237.g646b8c7d.el8.x86_64 31/32 2024-04-25T20:06:53.989 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1237.g646b8c7d.el8.x86_64 29/32 2024-04-25T20:06:53.989 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /sys 2024-04-25T20:06:53.989 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /proc 2024-04-25T20:06:53.989 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /mnt 2024-04-25T20:06:53.989 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /var/tmp 2024-04-25T20:06:53.989 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /home 2024-04-25T20:06:53.989 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /root 2024-04-25T20:06:53.989 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /tmp 2024-04-25T20:06:53.989 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:06:54.352 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-04-25T20:06:54.352 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:06:54.352 INFO:teuthology.orchestra.run.smithi039.stdout:Upgraded: 2024-04-25T20:06:54.352 INFO:teuthology.orchestra.run.smithi039.stdout: librados2-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:54.352 INFO:teuthology.orchestra.run.smithi039.stdout: librbd1-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:54.352 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-04-25T20:06:54.352 INFO:teuthology.orchestra.run.smithi039.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T20:06:54.352 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-base-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:54.352 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-common-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:54.353 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-radosgw-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:54.353 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-selinux-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:54.353 INFO:teuthology.orchestra.run.smithi039.stdout: fmt-8.1.1-5.el9.x86_64 2024-04-25T20:06:54.353 INFO:teuthology.orchestra.run.smithi039.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T20:06:54.353 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T20:06:54.353 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T20:06:54.353 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs2-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:54.353 INFO:teuthology.orchestra.run.smithi039.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T20:06:54.353 INFO:teuthology.orchestra.run.smithi039.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T20:06:54.353 INFO:teuthology.orchestra.run.smithi039.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T20:06:54.353 INFO:teuthology.orchestra.run.smithi039.stdout: libradosstriper1-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:54.353 INFO:teuthology.orchestra.run.smithi039.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T20:06:54.353 INFO:teuthology.orchestra.run.smithi039.stdout: librgw2-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:54.353 INFO:teuthology.orchestra.run.smithi039.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T20:06:54.354 INFO:teuthology.orchestra.run.smithi039.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T20:06:54.354 INFO:teuthology.orchestra.run.smithi039.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T20:06:54.354 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-argparse-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:54.354 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-common-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:54.354 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cephfs-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:54.354 INFO:teuthology.orchestra.run.smithi039.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T20:06:54.354 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rados-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:54.354 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rbd-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:54.354 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rgw-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:54.354 INFO:teuthology.orchestra.run.smithi039.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T20:06:54.354 INFO:teuthology.orchestra.run.smithi039.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T20:06:54.355 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:06:54.355 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-04-25T20:06:54.616 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-test 2024-04-25T20:06:54.774 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-radosgw-2:18.2.2-1237.g646b8c7d.el8.x86_64 30/32 2024-04-25T20:06:54.798 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1237.g646b8c7d.el8.x86_64 30/32 2024-04-25T20:06:54.798 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:06:54.799 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T20:06:54.799 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T20:06:54.799 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T20:06:54.799 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:06:54.801 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-25T20:06:54.885 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-25T20:06:54.885 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-25T20:06:55.160 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:00:28 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:06:55.286 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-04-25T20:06:55.286 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:06:55.287 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-04-25T20:06:55.287 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:06:55.287 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-04-25T20:06:55.287 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-test x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 40 M 2024-04-25T20:06:55.287 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2024-04-25T20:06:55.287 INFO:teuthology.orchestra.run.smithi039.stdout: libcephsqlite x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 166 k 2024-04-25T20:06:55.287 INFO:teuthology.orchestra.run.smithi039.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T20:06:55.287 INFO:teuthology.orchestra.run.smithi039.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T20:06:55.287 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:06:55.287 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-04-25T20:06:55.287 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:06:55.287 INFO:teuthology.orchestra.run.smithi039.stdout:Install 4 Packages 2024-04-25T20:06:55.287 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:06:55.288 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 40 M 2024-04-25T20:06:55.288 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 163 M 2024-04-25T20:06:55.288 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-04-25T20:06:55.602 INFO:teuthology.orchestra.run.smithi039.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.6 MB/s | 305 kB 00:00 2024-04-25T20:06:55.661 INFO:teuthology.orchestra.run.smithi039.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-25T20:06:55.696 INFO:teuthology.orchestra.run.smithi039.stdout:(3/4): libcephsqlite-18.2.2-1237.g646b8c7d.el8. 588 kB/s | 166 kB 00:00 2024-04-25T20:06:56.142 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-25T20:06:56.142 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-base-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/32 2024-04-25T20:06:56.142 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-common-2:18.2.2-1237.g646b8c7d.el8.x86_64 2/32 2024-04-25T20:06:56.142 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-radosgw-2:18.2.2-1237.g646b8c7d.el8.x86_64 3/32 2024-04-25T20:06:56.142 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-selinux-2:18.2.2-1237.g646b8c7d.el8.x86_64 4/32 2024-04-25T20:06:56.142 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephfs2-2:18.2.2-1237.g646b8c7d.el8.x86_64 5/32 2024-04-25T20:06:56.142 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libradosstriper1-2:18.2.2-1237.g646b8c7d.el8.x86_6 6/32 2024-04-25T20:06:56.143 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librgw2-2:18.2.2-1237.g646b8c7d.el8.x86_64 7/32 2024-04-25T20:06:56.143 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1237.g646b8c7d.el8. 8/32 2024-04-25T20:06:56.143 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ceph-common-2:18.2.2-1237.g646b8c7d.el8.x8 9/32 2024-04-25T20:06:56.143 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cephfs-2:18.2.2-1237.g646b8c7d.el8.x86_64 10/32 2024-04-25T20:06:56.143 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rados-2:18.2.2-1237.g646b8c7d.el8.x86_64 11/32 2024-04-25T20:06:56.143 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rbd-2:18.2.2-1237.g646b8c7d.el8.x86_64 12/32 2024-04-25T20:06:56.143 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rgw-2:18.2.2-1237.g646b8c7d.el8.x86_64 13/32 2024-04-25T20:06:56.143 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-04-25T20:06:56.143 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-04-25T20:06:56.143 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-04-25T20:06:56.143 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-04-25T20:06:56.143 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-04-25T20:06:56.143 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-04-25T20:06:56.143 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-04-25T20:06:56.144 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-04-25T20:06:56.144 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-04-25T20:06:56.144 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-04-25T20:06:56.144 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-04-25T20:06:56.144 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-04-25T20:06:56.144 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-04-25T20:06:56.144 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-04-25T20:06:56.144 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-04-25T20:06:56.144 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados2-2:18.2.2-1237.g646b8c7d.el8.x86_64 29/32 2024-04-25T20:06:56.144 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-04-25T20:06:56.145 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librbd1-2:18.2.2-1237.g646b8c7d.el8.x86_64 31/32 2024-04-25T20:06:56.908 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-04-25T20:06:56.908 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:06:56.908 INFO:teuthology.orchestra.run.smithi135.stdout:Upgraded: 2024-04-25T20:06:56.909 INFO:teuthology.orchestra.run.smithi135.stdout: librados2-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:56.909 INFO:teuthology.orchestra.run.smithi135.stdout: librbd1-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:56.909 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-04-25T20:06:56.909 INFO:teuthology.orchestra.run.smithi135.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T20:06:56.909 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-base-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:56.909 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-common-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:56.909 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-radosgw-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:56.909 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-selinux-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:56.909 INFO:teuthology.orchestra.run.smithi135.stdout: fmt-8.1.1-5.el9.x86_64 2024-04-25T20:06:56.909 INFO:teuthology.orchestra.run.smithi135.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T20:06:56.909 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T20:06:56.909 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T20:06:56.909 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs2-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:56.909 INFO:teuthology.orchestra.run.smithi135.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T20:06:56.909 INFO:teuthology.orchestra.run.smithi135.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T20:06:56.910 INFO:teuthology.orchestra.run.smithi135.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T20:06:56.910 INFO:teuthology.orchestra.run.smithi135.stdout: libradosstriper1-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:56.910 INFO:teuthology.orchestra.run.smithi135.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T20:06:56.910 INFO:teuthology.orchestra.run.smithi135.stdout: librgw2-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:56.910 INFO:teuthology.orchestra.run.smithi135.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T20:06:56.910 INFO:teuthology.orchestra.run.smithi135.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T20:06:56.910 INFO:teuthology.orchestra.run.smithi135.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T20:06:56.910 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-argparse-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:56.910 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-common-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:56.910 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cephfs-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:56.910 INFO:teuthology.orchestra.run.smithi135.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T20:06:56.910 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rados-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:56.910 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rbd-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:56.910 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rgw-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:06:56.911 INFO:teuthology.orchestra.run.smithi135.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T20:06:56.911 INFO:teuthology.orchestra.run.smithi135.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T20:06:56.911 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:06:56.911 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-25T20:06:57.175 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-test 2024-04-25T20:06:57.711 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:00:30 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:06:57.834 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-25T20:06:57.834 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:06:57.835 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-04-25T20:06:57.835 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:06:57.835 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-04-25T20:06:57.835 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-test x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 40 M 2024-04-25T20:06:57.835 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-04-25T20:06:57.835 INFO:teuthology.orchestra.run.smithi135.stdout: libcephsqlite x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 166 k 2024-04-25T20:06:57.835 INFO:teuthology.orchestra.run.smithi135.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T20:06:57.835 INFO:teuthology.orchestra.run.smithi135.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T20:06:57.835 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:06:57.835 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-04-25T20:06:57.835 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:06:57.835 INFO:teuthology.orchestra.run.smithi135.stdout:Install 4 Packages 2024-04-25T20:06:57.835 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:06:57.836 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 40 M 2024-04-25T20:06:57.836 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 163 M 2024-04-25T20:06:57.836 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-04-25T20:06:58.097 INFO:teuthology.orchestra.run.smithi135.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.6 MB/s | 305 kB 00:00 2024-04-25T20:06:58.148 INFO:teuthology.orchestra.run.smithi135.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T20:06:58.443 INFO:teuthology.orchestra.run.smithi135.stdout:(3/4): libcephsqlite-18.2.2-1237.g646b8c7d.el8. 310 kB/s | 166 kB 00:00 2024-04-25T20:06:58.915 INFO:teuthology.orchestra.run.smithi039.stdout:(4/4): ceph-test-18.2.2-1237.g646b8c7d.el8.x86_ 11 MB/s | 40 MB 00:03 2024-04-25T20:06:58.918 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:06:58.918 INFO:teuthology.orchestra.run.smithi039.stdout:Total 11 MB/s | 40 MB 00:03 2024-04-25T20:06:58.945 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-04-25T20:06:58.967 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-04-25T20:06:58.967 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-04-25T20:06:59.299 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-04-25T20:06:59.299 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-04-25T20:06:59.782 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-04-25T20:06:59.866 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-25T20:06:59.914 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-25T20:06:59.941 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephsqlite-2:18.2.2-1237.g646b8c7d.el8.x86_64 3/4 2024-04-25T20:07:00.713 INFO:teuthology.orchestra.run.smithi135.stdout:(4/4): ceph-test-18.2.2-1237.g646b8c7d.el8.x86_ 14 MB/s | 40 MB 00:02 2024-04-25T20:07:00.716 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:07:00.716 INFO:teuthology.orchestra.run.smithi135.stdout:Total 14 MB/s | 40 MB 00:02 2024-04-25T20:07:00.742 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-04-25T20:07:00.761 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-04-25T20:07:00.762 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-04-25T20:07:01.075 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-04-25T20:07:01.075 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-04-25T20:07:01.538 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-04-25T20:07:01.613 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-25T20:07:01.680 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-25T20:07:01.706 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephsqlite-2:18.2.2-1237.g646b8c7d.el8.x86_64 3/4 2024-04-25T20:07:03.821 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1237.g646b8c7d.el8.x86_64 3/4 2024-04-25T20:07:03.899 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-test-2:18.2.2-1237.g646b8c7d.el8.x86_64 4/4 2024-04-25T20:07:05.376 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1237.g646b8c7d.el8.x86_64 3/4 2024-04-25T20:07:05.462 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-test-2:18.2.2-1237.g646b8c7d.el8.x86_64 4/4 2024-04-25T20:07:05.877 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-test-2:18.2.2-1237.g646b8c7d.el8.x86_64 4/4 2024-04-25T20:07:05.877 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-test-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/4 2024-04-25T20:07:05.877 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephsqlite-2:18.2.2-1237.g646b8c7d.el8.x86_64 2/4 2024-04-25T20:07:05.877 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-25T20:07:06.152 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-25T20:07:06.152 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:06.153 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-04-25T20:07:06.153 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-test-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:07:06.153 INFO:teuthology.orchestra.run.smithi039.stdout: libcephsqlite-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:07:06.153 INFO:teuthology.orchestra.run.smithi039.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T20:07:06.153 INFO:teuthology.orchestra.run.smithi039.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T20:07:06.153 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:06.153 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-04-25T20:07:06.526 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph 2024-04-25T20:07:07.055 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:00:40 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:07:07.181 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-04-25T20:07:07.184 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:07:07.184 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-04-25T20:07:07.184 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:07:07.184 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-04-25T20:07:07.184 INFO:teuthology.orchestra.run.smithi039.stdout: ceph x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 6.5 k 2024-04-25T20:07:07.184 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2024-04-25T20:07:07.184 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mds x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 2.1 M 2024-04-25T20:07:07.184 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 1.5 M 2024-04-25T20:07:07.184 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1237.g646b8c7d.el8 ceph-noarch 246 k 2024-04-25T20:07:07.184 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mon x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 4.7 M 2024-04-25T20:07:07.184 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-osd x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 17 M 2024-04-25T20:07:07.184 INFO:teuthology.orchestra.run.smithi039.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T20:07:07.184 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T20:07:07.185 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T20:07:07.185 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T20:07:07.185 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T20:07:07.185 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T20:07:07.185 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T20:07:07.185 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-collections 2024-04-25T20:07:07.185 INFO:teuthology.orchestra.run.smithi039.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-04-25T20:07:07.185 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T20:07:07.185 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T20:07:07.185 INFO:teuthology.orchestra.run.smithi039.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T20:07:07.185 INFO:teuthology.orchestra.run.smithi039.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T20:07:07.185 INFO:teuthology.orchestra.run.smithi039.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T20:07:07.185 INFO:teuthology.orchestra.run.smithi039.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T20:07:07.185 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T20:07:07.186 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T20:07:07.186 INFO:teuthology.orchestra.run.smithi039.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T20:07:07.186 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T20:07:07.186 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T20:07:07.186 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T20:07:07.186 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T20:07:07.186 INFO:teuthology.orchestra.run.smithi039.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T20:07:07.186 INFO:teuthology.orchestra.run.smithi039.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T20:07:07.186 INFO:teuthology.orchestra.run.smithi039.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T20:07:07.186 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T20:07:07.186 INFO:teuthology.orchestra.run.smithi039.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T20:07:07.186 INFO:teuthology.orchestra.run.smithi039.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T20:07:07.186 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:07.186 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-04-25T20:07:07.187 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:07:07.187 INFO:teuthology.orchestra.run.smithi039.stdout:Install 33 Packages 2024-04-25T20:07:07.187 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:07.187 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 29 M 2024-04-25T20:07:07.187 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 104 M 2024-04-25T20:07:07.187 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-04-25T20:07:07.598 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-test-2:18.2.2-1237.g646b8c7d.el8.x86_64 4/4 2024-04-25T20:07:07.598 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-test-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/4 2024-04-25T20:07:07.598 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephsqlite-2:18.2.2-1237.g646b8c7d.el8.x86_64 2/4 2024-04-25T20:07:07.598 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-25T20:07:07.849 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-25T20:07:07.850 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:07.850 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-04-25T20:07:07.850 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-test-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:07:07.850 INFO:teuthology.orchestra.run.smithi135.stdout: libcephsqlite-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:07:07.850 INFO:teuthology.orchestra.run.smithi135.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T20:07:07.850 INFO:teuthology.orchestra.run.smithi135.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T20:07:07.850 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:07.850 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-25T20:07:07.998 INFO:teuthology.orchestra.run.smithi039.stdout:(1/33): ceph-18.2.2-1237.g646b8c7d.el8.x86_64.r 26 kB/s | 6.5 kB 00:00 2024-04-25T20:07:08.140 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph 2024-04-25T20:07:08.148 INFO:teuthology.orchestra.run.smithi039.stdout:(2/33): ceph-mds-18.2.2-1237.g646b8c7d.el8.x86_ 5.2 MB/s | 2.1 MB 00:00 2024-04-25T20:07:08.415 INFO:teuthology.orchestra.run.smithi039.stdout:(3/33): ceph-mgr-18.2.2-1237.g646b8c7d.el8.x86_ 2.2 MB/s | 1.5 MB 00:00 2024-04-25T20:07:08.541 INFO:teuthology.orchestra.run.smithi039.stdout:(4/33): ceph-mgr-modules-core-18.2.2-1237.g646b 1.9 MB/s | 246 kB 00:00 2024-04-25T20:07:08.656 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:00:41 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:07:08.674 INFO:teuthology.orchestra.run.smithi039.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 253 kB 00:00 2024-04-25T20:07:08.749 INFO:teuthology.orchestra.run.smithi039.stdout:(6/33): python3-cryptography-36.0.1-4.el9.x86_6 16 MB/s | 1.2 MB 00:00 2024-04-25T20:07:08.782 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-25T20:07:08.783 INFO:teuthology.orchestra.run.smithi039.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 3.1 MB/s | 106 kB 00:00 2024-04-25T20:07:08.785 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:07:08.785 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-04-25T20:07:08.785 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:07:08.785 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-04-25T20:07:08.785 INFO:teuthology.orchestra.run.smithi135.stdout: ceph x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 6.5 k 2024-04-25T20:07:08.786 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-04-25T20:07:08.786 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mds x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 2.1 M 2024-04-25T20:07:08.786 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 1.5 M 2024-04-25T20:07:08.786 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1237.g646b8c7d.el8 ceph-noarch 246 k 2024-04-25T20:07:08.786 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mon x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 4.7 M 2024-04-25T20:07:08.786 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-osd x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 17 M 2024-04-25T20:07:08.786 INFO:teuthology.orchestra.run.smithi135.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T20:07:08.786 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T20:07:08.786 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T20:07:08.786 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T20:07:08.786 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T20:07:08.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T20:07:08.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T20:07:08.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-collections 2024-04-25T20:07:08.787 INFO:teuthology.orchestra.run.smithi135.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-04-25T20:07:08.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T20:07:08.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T20:07:08.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T20:07:08.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T20:07:08.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T20:07:08.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T20:07:08.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T20:07:08.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T20:07:08.788 INFO:teuthology.orchestra.run.smithi135.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T20:07:08.788 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T20:07:08.788 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T20:07:08.788 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T20:07:08.788 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T20:07:08.788 INFO:teuthology.orchestra.run.smithi135.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T20:07:08.788 INFO:teuthology.orchestra.run.smithi135.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T20:07:08.788 INFO:teuthology.orchestra.run.smithi135.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T20:07:08.788 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T20:07:08.788 INFO:teuthology.orchestra.run.smithi135.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T20:07:08.788 INFO:teuthology.orchestra.run.smithi135.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T20:07:08.788 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:08.788 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-04-25T20:07:08.788 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:07:08.789 INFO:teuthology.orchestra.run.smithi135.stdout:Install 33 Packages 2024-04-25T20:07:08.789 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:08.789 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 29 M 2024-04-25T20:07:08.789 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 104 M 2024-04-25T20:07:08.789 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-04-25T20:07:08.816 INFO:teuthology.orchestra.run.smithi039.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 4.0 MB/s | 135 kB 00:00 2024-04-25T20:07:09.008 INFO:teuthology.orchestra.run.smithi039.stdout:(9/33): ceph-osd-18.2.2-1237.g646b8c7d.el8.x86_ 19 MB/s | 17 MB 00:00 2024-04-25T20:07:09.034 INFO:teuthology.orchestra.run.smithi039.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 575 kB/s | 125 kB 00:00 2024-04-25T20:07:09.059 INFO:teuthology.orchestra.run.smithi039.stdout:(11/33): python3-urllib3-1.26.5-5.el9.noarch.rp 4.2 MB/s | 215 kB 00:00 2024-04-25T20:07:09.084 INFO:teuthology.orchestra.run.smithi039.stdout:(12/33): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-25T20:07:09.109 INFO:teuthology.orchestra.run.smithi039.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 697 kB/s | 35 kB 00:00 2024-04-25T20:07:09.139 INFO:teuthology.orchestra.run.smithi039.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 942 kB/s | 51 kB 00:00 2024-04-25T20:07:09.160 INFO:teuthology.orchestra.run.smithi039.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 831 kB/s | 42 kB 00:00 2024-04-25T20:07:09.260 INFO:teuthology.orchestra.run.smithi039.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 359 kB/s | 43 kB 00:00 2024-04-25T20:07:09.285 INFO:teuthology.orchestra.run.smithi039.stdout:(17/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-04-25T20:07:09.327 INFO:teuthology.orchestra.run.smithi039.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 256 kB/s | 11 kB 00:00 2024-04-25T20:07:09.369 INFO:teuthology.orchestra.run.smithi039.stdout:(19/33): python3-cherrypy-18.6.1-2.el9.noarch.r 3.2 MB/s | 358 kB 00:00 2024-04-25T20:07:09.395 INFO:teuthology.orchestra.run.smithi039.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 264 kB/s | 18 kB 00:00 2024-04-25T20:07:09.420 INFO:teuthology.orchestra.run.smithi039.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 464 kB/s | 23 kB 00:00 2024-04-25T20:07:09.445 INFO:teuthology.orchestra.run.smithi039.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 388 kB/s | 19 kB 00:00 2024-04-25T20:07:09.470 INFO:teuthology.orchestra.run.smithi039.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-04-25T20:07:09.496 INFO:teuthology.orchestra.run.smithi039.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 922 kB/s | 46 kB 00:00 2024-04-25T20:07:09.521 INFO:teuthology.orchestra.run.smithi039.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-04-25T20:07:09.555 INFO:teuthology.orchestra.run.smithi039.stdout:(26/33): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2024-04-25T20:07:09.569 INFO:teuthology.orchestra.run.smithi135.stdout:(1/33): ceph-18.2.2-1237.g646b8c7d.el8.x86_64.r 27 kB/s | 6.5 kB 00:00 2024-04-25T20:07:09.571 INFO:teuthology.orchestra.run.smithi039.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-04-25T20:07:09.655 INFO:teuthology.orchestra.run.smithi039.stdout:(28/33): ceph-mon-18.2.2-1237.g646b8c7d.el8.x86 2.8 MB/s | 4.7 MB 00:01 2024-04-25T20:07:09.680 INFO:teuthology.orchestra.run.smithi039.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 718 kB/s | 90 kB 00:00 2024-04-25T20:07:09.697 INFO:teuthology.orchestra.run.smithi039.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 285 kB/s | 36 kB 00:00 2024-04-25T20:07:09.710 INFO:teuthology.orchestra.run.smithi135.stdout:(2/33): ceph-mds-18.2.2-1237.g646b8c7d.el8.x86_ 5.5 MB/s | 2.1 MB 00:00 2024-04-25T20:07:09.731 INFO:teuthology.orchestra.run.smithi039.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 606 kB/s | 20 kB 00:00 2024-04-25T20:07:09.781 INFO:teuthology.orchestra.run.smithi039.stdout:(32/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 5.0 MB/s | 427 kB 00:00 2024-04-25T20:07:09.814 INFO:teuthology.orchestra.run.smithi039.stdout:(33/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.4 MB/s | 230 kB 00:00 2024-04-25T20:07:09.819 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:07:09.819 INFO:teuthology.orchestra.run.smithi039.stdout:Total 11 MB/s | 29 MB 00:02 2024-04-25T20:07:09.885 INFO:teuthology.orchestra.run.smithi135.stdout:(3/33): ceph-mgr-18.2.2-1237.g646b8c7d.el8.x86_ 2.6 MB/s | 1.5 MB 00:00 2024-04-25T20:07:09.943 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-04-25T20:07:09.986 INFO:teuthology.orchestra.run.smithi135.stdout:(4/33): ceph-mgr-modules-core-18.2.2-1237.g646b 2.4 MB/s | 246 kB 00:00 2024-04-25T20:07:10.008 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-04-25T20:07:10.008 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-04-25T20:07:10.103 INFO:teuthology.orchestra.run.smithi135.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-04-25T20:07:10.179 INFO:teuthology.orchestra.run.smithi135.stdout:(6/33): python3-cryptography-36.0.1-4.el9.x86_6 16 MB/s | 1.2 MB 00:00 2024-04-25T20:07:10.212 INFO:teuthology.orchestra.run.smithi135.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 3.1 MB/s | 106 kB 00:00 2024-04-25T20:07:10.246 INFO:teuthology.orchestra.run.smithi135.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 4.0 MB/s | 135 kB 00:00 2024-04-25T20:07:10.280 INFO:teuthology.orchestra.run.smithi135.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 3.6 MB/s | 125 kB 00:00 2024-04-25T20:07:10.322 INFO:teuthology.orchestra.run.smithi135.stdout:(10/33): python3-urllib3-1.26.5-5.el9.noarch.rp 5.0 MB/s | 215 kB 00:00 2024-04-25T20:07:10.322 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-04-25T20:07:10.323 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-04-25T20:07:10.364 INFO:teuthology.orchestra.run.smithi135.stdout:(11/33): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-04-25T20:07:10.397 INFO:teuthology.orchestra.run.smithi135.stdout:(12/33): python3-markupsafe-1.1.1-12.el9.x86_64 1.0 MB/s | 35 kB 00:00 2024-04-25T20:07:10.564 INFO:teuthology.orchestra.run.smithi135.stdout:(13/33): ceph-osd-18.2.2-1237.g646b8c7d.el8.x86 19 MB/s | 17 MB 00:00 2024-04-25T20:07:10.590 INFO:teuthology.orchestra.run.smithi135.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 265 kB/s | 51 kB 00:00 2024-04-25T20:07:10.615 INFO:teuthology.orchestra.run.smithi135.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 832 kB/s | 42 kB 00:00 2024-04-25T20:07:10.990 INFO:teuthology.orchestra.run.smithi135.stdout:(16/33): ceph-mon-18.2.2-1237.g646b8c7d.el8.x86 3.3 MB/s | 4.7 MB 00:01 2024-04-25T20:07:11.016 INFO:teuthology.orchestra.run.smithi135.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 102 kB/s | 43 kB 00:00 2024-04-25T20:07:11.166 INFO:teuthology.orchestra.run.smithi135.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 312 kB/s | 172 kB 00:00 2024-04-25T20:07:11.208 INFO:teuthology.orchestra.run.smithi135.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 56 kB/s | 11 kB 00:00 2024-04-25T20:07:11.433 INFO:teuthology.orchestra.run.smithi135.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 66 kB/s | 18 kB 00:00 2024-04-25T20:07:11.492 INFO:teuthology.orchestra.run.smithi135.stdout:(21/33): python3-cherrypy-18.6.1-2.el9.noarch.r 715 kB/s | 358 kB 00:00 2024-04-25T20:07:11.517 INFO:teuthology.orchestra.run.smithi135.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 75 kB/s | 23 kB 00:00 2024-04-25T20:07:11.626 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-04-25T20:07:11.701 INFO:teuthology.orchestra.run.smithi135.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 73 kB/s | 19 kB 00:00 2024-04-25T20:07:11.702 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-25T20:07:11.734 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-25T20:07:11.759 INFO:teuthology.orchestra.run.smithi135.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 74 kB/s | 20 kB 00:00 2024-04-25T20:07:11.775 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-25T20:07:11.860 INFO:teuthology.orchestra.run.smithi135.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 135 kB/s | 46 kB 00:00 2024-04-25T20:07:11.872 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-25T20:07:11.949 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-25T20:07:11.994 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-25T20:07:12.030 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-25T20:07:12.043 INFO:teuthology.orchestra.run.smithi135.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 230 kB/s | 79 kB 00:00 2024-04-25T20:07:12.059 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-25T20:07:12.093 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-25T20:07:12.127 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-25T20:07:12.127 INFO:teuthology.orchestra.run.smithi135.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 61 kB/s | 16 kB 00:00 2024-04-25T20:07:12.168 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-25T20:07:12.211 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-25T20:07:12.245 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-25T20:07:12.283 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-25T20:07:12.369 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-25T20:07:12.385 INFO:teuthology.orchestra.run.smithi135.stdout:(28/33): python3-pecan-1.4.2-3.el9.noarch.rpm 435 kB/s | 272 kB 00:00 2024-04-25T20:07:12.411 INFO:teuthology.orchestra.run.smithi135.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 126 kB/s | 36 kB 00:00 2024-04-25T20:07:12.437 INFO:teuthology.orchestra.run.smithi135.stdout:(30/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 229 kB/s | 90 kB 00:00 2024-04-25T20:07:12.649 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-25T20:07:12.696 INFO:teuthology.orchestra.run.smithi135.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 77 kB/s | 20 kB 00:00 2024-04-25T20:07:12.729 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-25T20:07:12.778 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-25T20:07:12.817 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-25T20:07:12.875 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-25T20:07:12.929 INFO:teuthology.orchestra.run.smithi135.stdout:(32/33): python3-webob-1.8.7-6.el9.noarch.rpm 423 kB/s | 230 kB 00:00 2024-04-25T20:07:12.938 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-25T20:07:13.037 INFO:teuthology.orchestra.run.smithi135.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 683 kB/s | 427 kB 00:00 2024-04-25T20:07:13.042 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:07:13.042 INFO:teuthology.orchestra.run.smithi135.stdout:Total 6.9 MB/s | 29 MB 00:04 2024-04-25T20:07:13.138 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-25T20:07:13.160 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-04-25T20:07:13.195 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-25T20:07:13.198 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-04-25T20:07:13.198 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-04-25T20:07:13.286 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-25T20:07:13.374 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-25T20:07:13.419 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-25T20:07:13.480 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-04-25T20:07:13.480 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-04-25T20:07:13.541 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-25T20:07:13.709 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1237.g646b8c7d.el8. 28/33 2024-04-25T20:07:13.737 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-2:18.2.2-1237.g646b8c7d.el8.x86_64 29/33 2024-04-25T20:07:13.764 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1237.g646b8c7d.el8.x86_64 29/33 2024-04-25T20:07:13.765 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:07:13.765 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T20:07:13.765 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T20:07:13.765 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T20:07:13.765 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:14.664 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-04-25T20:07:14.724 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-25T20:07:14.757 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-25T20:07:14.790 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-25T20:07:14.882 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-25T20:07:14.970 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-25T20:07:15.010 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-25T20:07:15.054 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-25T20:07:15.085 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-25T20:07:15.118 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-25T20:07:15.152 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-25T20:07:15.193 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-25T20:07:15.236 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-25T20:07:15.255 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-osd-2:18.2.2-1237.g646b8c7d.el8.x86_64 30/33 2024-04-25T20:07:15.270 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-25T20:07:15.283 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-osd-2:18.2.2-1237.g646b8c7d.el8.x86_64 30/33 2024-04-25T20:07:15.283 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:07:15.283 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T20:07:15.283 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T20:07:15.283 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T20:07:15.283 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:15.309 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-25T20:07:15.383 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-25T20:07:15.540 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-25T20:07:15.619 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-25T20:07:15.672 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-25T20:07:15.719 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-25T20:07:15.771 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mon-2:18.2.2-1237.g646b8c7d.el8.x86_64 31/33 2024-04-25T20:07:15.776 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-25T20:07:15.795 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mon-2:18.2.2-1237.g646b8c7d.el8.x86_64 31/33 2024-04-25T20:07:15.796 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:07:15.796 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T20:07:15.796 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T20:07:15.796 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T20:07:15.796 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:15.830 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-25T20:07:16.014 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mds-2:18.2.2-1237.g646b8c7d.el8.x86_64 32/33 2024-04-25T20:07:16.015 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-25T20:07:16.039 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mds-2:18.2.2-1237.g646b8c7d.el8.x86_64 32/33 2024-04-25T20:07:16.039 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:07:16.039 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T20:07:16.039 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T20:07:16.039 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T20:07:16.039 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:16.064 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-25T20:07:16.103 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-2:18.2.2-1237.g646b8c7d.el8.x86_64 33/33 2024-04-25T20:07:16.135 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-25T20:07:16.231 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-25T20:07:16.273 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-25T20:07:16.389 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-25T20:07:16.548 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1237.g646b8c7d.el8. 28/33 2024-04-25T20:07:16.576 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-2:18.2.2-1237.g646b8c7d.el8.x86_64 29/33 2024-04-25T20:07:16.602 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1237.g646b8c7d.el8.x86_64 29/33 2024-04-25T20:07:16.602 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:07:16.602 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T20:07:16.602 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T20:07:16.602 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T20:07:16.602 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:17.804 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-2:18.2.2-1237.g646b8c7d.el8.x86_64 33/33 2024-04-25T20:07:17.804 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/33 2024-04-25T20:07:17.804 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mds-2:18.2.2-1237.g646b8c7d.el8.x86_64 2/33 2024-04-25T20:07:17.804 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-2:18.2.2-1237.g646b8c7d.el8.x86_64 3/33 2024-04-25T20:07:17.804 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mon-2:18.2.2-1237.g646b8c7d.el8.x86_64 4/33 2024-04-25T20:07:17.804 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-osd-2:18.2.2-1237.g646b8c7d.el8.x86_64 5/33 2024-04-25T20:07:17.804 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1237.g646b8c7d.el8. 6/33 2024-04-25T20:07:17.804 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-25T20:07:17.804 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-25T20:07:17.804 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-25T20:07:17.805 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-25T20:07:17.805 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-25T20:07:17.805 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-25T20:07:17.805 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-25T20:07:17.805 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-25T20:07:17.805 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-25T20:07:17.805 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-25T20:07:17.805 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-25T20:07:17.805 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-25T20:07:17.805 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-25T20:07:17.805 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-25T20:07:17.805 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-25T20:07:17.805 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-25T20:07:17.805 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-25T20:07:17.805 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-25T20:07:17.806 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-25T20:07:17.806 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-25T20:07:17.806 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-25T20:07:17.806 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-25T20:07:17.806 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-25T20:07:17.806 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-25T20:07:17.806 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-25T20:07:17.806 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-25T20:07:18.019 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-osd-2:18.2.2-1237.g646b8c7d.el8.x86_64 30/33 2024-04-25T20:07:18.046 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-osd-2:18.2.2-1237.g646b8c7d.el8.x86_64 30/33 2024-04-25T20:07:18.046 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:07:18.046 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T20:07:18.046 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T20:07:18.046 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T20:07:18.046 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:18.487 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mon-2:18.2.2-1237.g646b8c7d.el8.x86_64 31/33 2024-04-25T20:07:18.510 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mon-2:18.2.2-1237.g646b8c7d.el8.x86_64 31/33 2024-04-25T20:07:18.510 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:07:18.510 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T20:07:18.511 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T20:07:18.511 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T20:07:18.511 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:18.713 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mds-2:18.2.2-1237.g646b8c7d.el8.x86_64 32/33 2024-04-25T20:07:18.735 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mds-2:18.2.2-1237.g646b8c7d.el8.x86_64 32/33 2024-04-25T20:07:18.736 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:07:18.736 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T20:07:18.736 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T20:07:18.736 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T20:07:18.736 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:18.792 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-2:18.2.2-1237.g646b8c7d.el8.x86_64 33/33 2024-04-25T20:07:18.815 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-25T20:07:18.815 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:18.815 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-04-25T20:07:18.815 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:07:18.815 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mds-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:07:18.815 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:07:18.815 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-modules-core-2:18.2.2-1237.g646b8c7d.el8.noarch 2024-04-25T20:07:18.815 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mon-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:07:18.815 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-osd-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:07:18.816 INFO:teuthology.orchestra.run.smithi039.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T20:07:18.816 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T20:07:18.816 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T20:07:18.816 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T20:07:18.816 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T20:07:18.816 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T20:07:18.816 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T20:07:18.816 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T20:07:18.816 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T20:07:18.816 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T20:07:18.816 INFO:teuthology.orchestra.run.smithi039.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T20:07:18.816 INFO:teuthology.orchestra.run.smithi039.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T20:07:18.816 INFO:teuthology.orchestra.run.smithi039.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T20:07:18.816 INFO:teuthology.orchestra.run.smithi039.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T20:07:18.816 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T20:07:18.817 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T20:07:18.817 INFO:teuthology.orchestra.run.smithi039.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T20:07:18.817 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T20:07:18.817 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T20:07:18.817 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T20:07:18.817 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T20:07:18.817 INFO:teuthology.orchestra.run.smithi039.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T20:07:18.817 INFO:teuthology.orchestra.run.smithi039.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T20:07:18.817 INFO:teuthology.orchestra.run.smithi039.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T20:07:18.817 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T20:07:18.817 INFO:teuthology.orchestra.run.smithi039.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T20:07:18.817 INFO:teuthology.orchestra.run.smithi039.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T20:07:18.817 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:18.817 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-04-25T20:07:19.134 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-base 2024-04-25T20:07:19.671 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:00:52 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:07:19.752 INFO:teuthology.orchestra.run.smithi039.stdout:Package ceph-base-2:18.2.2-1237.g646b8c7d.el8.x86_64 is already installed. 2024-04-25T20:07:19.796 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-04-25T20:07:19.797 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2024-04-25T20:07:19.797 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-04-25T20:07:19.858 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install cephadm 2024-04-25T20:07:20.394 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-2:18.2.2-1237.g646b8c7d.el8.x86_64 33/33 2024-04-25T20:07:20.394 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/33 2024-04-25T20:07:20.394 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mds-2:18.2.2-1237.g646b8c7d.el8.x86_64 2/33 2024-04-25T20:07:20.394 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-2:18.2.2-1237.g646b8c7d.el8.x86_64 3/33 2024-04-25T20:07:20.394 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mon-2:18.2.2-1237.g646b8c7d.el8.x86_64 4/33 2024-04-25T20:07:20.394 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-osd-2:18.2.2-1237.g646b8c7d.el8.x86_64 5/33 2024-04-25T20:07:20.394 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1237.g646b8c7d.el8. 6/33 2024-04-25T20:07:20.394 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-25T20:07:20.395 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-25T20:07:20.395 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-25T20:07:20.395 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-25T20:07:20.395 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-25T20:07:20.395 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-25T20:07:20.395 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-25T20:07:20.395 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-25T20:07:20.395 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-25T20:07:20.395 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-25T20:07:20.395 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-25T20:07:20.395 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-25T20:07:20.395 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-25T20:07:20.395 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-25T20:07:20.395 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-25T20:07:20.395 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-25T20:07:20.395 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-25T20:07:20.396 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-25T20:07:20.396 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-25T20:07:20.396 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-25T20:07:20.396 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-25T20:07:20.396 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-25T20:07:20.396 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:00:53 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:07:20.397 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-25T20:07:20.397 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-25T20:07:20.397 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-25T20:07:20.397 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-25T20:07:20.518 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-04-25T20:07:20.519 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:07:20.519 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-04-25T20:07:20.519 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:07:20.519 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-04-25T20:07:20.519 INFO:teuthology.orchestra.run.smithi039.stdout: cephadm noarch 2:18.2.2-1237.g646b8c7d.el8 ceph-noarch 224 k 2024-04-25T20:07:20.519 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:20.519 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-04-25T20:07:20.519 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:07:20.519 INFO:teuthology.orchestra.run.smithi039.stdout:Install 1 Package 2024-04-25T20:07:20.519 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:20.520 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 224 k 2024-04-25T20:07:20.520 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 216 k 2024-04-25T20:07:20.520 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-04-25T20:07:20.871 INFO:teuthology.orchestra.run.smithi039.stdout:cephadm-18.2.2-1237.g646b8c7d.el8.noarch.rpm 640 kB/s | 224 kB 00:00 2024-04-25T20:07:20.871 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:07:20.871 INFO:teuthology.orchestra.run.smithi039.stdout:Total 637 kB/s | 224 kB 00:00 2024-04-25T20:07:20.872 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-04-25T20:07:20.877 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-04-25T20:07:20.877 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-04-25T20:07:20.880 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-04-25T20:07:20.880 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-04-25T20:07:20.950 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-04-25T20:07:21.417 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: cephadm-2:18.2.2-1237.g646b8c7d.el8.noarch 1/1 2024-04-25T20:07:21.441 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-25T20:07:21.442 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:21.442 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-04-25T20:07:21.442 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:07:21.442 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mds-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:07:21.442 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:07:21.442 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-modules-core-2:18.2.2-1237.g646b8c7d.el8.noarch 2024-04-25T20:07:21.442 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mon-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:07:21.442 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-osd-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:07:21.442 INFO:teuthology.orchestra.run.smithi135.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T20:07:21.442 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T20:07:21.442 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T20:07:21.442 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T20:07:21.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T20:07:21.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T20:07:21.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T20:07:21.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T20:07:21.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T20:07:21.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T20:07:21.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T20:07:21.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T20:07:21.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T20:07:21.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T20:07:21.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T20:07:21.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T20:07:21.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T20:07:21.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T20:07:21.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T20:07:21.444 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T20:07:21.444 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T20:07:21.444 INFO:teuthology.orchestra.run.smithi135.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T20:07:21.444 INFO:teuthology.orchestra.run.smithi135.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T20:07:21.444 INFO:teuthology.orchestra.run.smithi135.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T20:07:21.444 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T20:07:21.444 INFO:teuthology.orchestra.run.smithi135.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T20:07:21.444 INFO:teuthology.orchestra.run.smithi135.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T20:07:21.444 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:21.444 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-25T20:07:21.500 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephadm-2:18.2.2-1237.g646b8c7d.el8.noarch 1/1 2024-04-25T20:07:21.795 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-base 2024-04-25T20:07:21.812 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: cephadm-2:18.2.2-1237.g646b8c7d.el8.noarch 1/1 2024-04-25T20:07:22.035 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephadm-2:18.2.2-1237.g646b8c7d.el8.noarch 1/1 2024-04-25T20:07:22.036 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:22.036 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-04-25T20:07:22.036 INFO:teuthology.orchestra.run.smithi039.stdout: cephadm-2:18.2.2-1237.g646b8c7d.el8.noarch 2024-04-25T20:07:22.036 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:22.036 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-04-25T20:07:22.245 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-immutable-object-cache 2024-04-25T20:07:22.315 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:00:55 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:07:22.395 INFO:teuthology.orchestra.run.smithi135.stdout:Package ceph-base-2:18.2.2-1237.g646b8c7d.el8.x86_64 is already installed. 2024-04-25T20:07:22.435 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-25T20:07:22.436 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2024-04-25T20:07:22.436 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-25T20:07:22.500 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install cephadm 2024-04-25T20:07:22.785 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:00:55 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:07:22.912 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-04-25T20:07:22.913 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:07:22.913 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repo Size 2024-04-25T20:07:22.913 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:07:22.913 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-04-25T20:07:22.913 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 143 k 2024-04-25T20:07:22.913 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:22.914 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-04-25T20:07:22.914 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:07:22.914 INFO:teuthology.orchestra.run.smithi039.stdout:Install 1 Package 2024-04-25T20:07:22.914 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:22.914 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 143 k 2024-04-25T20:07:22.914 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 395 k 2024-04-25T20:07:22.914 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-04-25T20:07:23.007 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:00:56 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:07:23.125 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-25T20:07:23.126 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:07:23.126 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-04-25T20:07:23.126 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:07:23.126 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-04-25T20:07:23.126 INFO:teuthology.orchestra.run.smithi135.stdout: cephadm noarch 2:18.2.2-1237.g646b8c7d.el8 ceph-noarch 224 k 2024-04-25T20:07:23.126 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:23.126 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-04-25T20:07:23.126 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:07:23.126 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2024-04-25T20:07:23.126 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:23.126 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 224 k 2024-04-25T20:07:23.126 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 216 k 2024-04-25T20:07:23.127 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-04-25T20:07:23.135 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-immutable-object-cache-18.2.2-1237.g646b8c 644 kB/s | 143 kB 00:00 2024-04-25T20:07:23.135 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:07:23.136 INFO:teuthology.orchestra.run.smithi039.stdout:Total 639 kB/s | 143 kB 00:00 2024-04-25T20:07:23.136 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-04-25T20:07:23.144 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-04-25T20:07:23.145 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-04-25T20:07:23.205 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-04-25T20:07:23.206 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-04-25T20:07:23.359 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-04-25T20:07:23.444 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1237.g646b8c7d. 1/1 2024-04-25T20:07:23.470 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1237.g646b8c7d. 1/1 2024-04-25T20:07:23.470 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:07:23.470 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T20:07:23.470 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:24.008 INFO:teuthology.orchestra.run.smithi135.stdout:cephadm-18.2.2-1237.g646b8c7d.el8.noarch.rpm 254 kB/s | 224 kB 00:00 2024-04-25T20:07:24.009 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:07:24.009 INFO:teuthology.orchestra.run.smithi135.stdout:Total 254 kB/s | 224 kB 00:00 2024-04-25T20:07:24.009 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-04-25T20:07:24.014 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-04-25T20:07:24.014 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-04-25T20:07:24.017 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-04-25T20:07:24.017 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-04-25T20:07:24.083 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-04-25T20:07:24.326 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1237.g646b8c7d. 1/1 2024-04-25T20:07:24.326 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:24.326 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-04-25T20:07:24.326 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-immutable-object-cache-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:07:24.326 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:24.326 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-04-25T20:07:24.512 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: cephadm-2:18.2.2-1237.g646b8c7d.el8.noarch 1/1 2024-04-25T20:07:24.521 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-mgr 2024-04-25T20:07:24.603 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephadm-2:18.2.2-1237.g646b8c7d.el8.noarch 1/1 2024-04-25T20:07:24.946 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: cephadm-2:18.2.2-1237.g646b8c7d.el8.noarch 1/1 2024-04-25T20:07:25.056 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:00:58 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:07:25.139 INFO:teuthology.orchestra.run.smithi039.stdout:Package ceph-mgr-2:18.2.2-1237.g646b8c7d.el8.x86_64 is already installed. 2024-04-25T20:07:25.182 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephadm-2:18.2.2-1237.g646b8c7d.el8.noarch 1/1 2024-04-25T20:07:25.182 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:25.182 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-04-25T20:07:25.182 INFO:teuthology.orchestra.run.smithi135.stdout: cephadm-2:18.2.2-1237.g646b8c7d.el8.noarch 2024-04-25T20:07:25.182 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:25.182 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-25T20:07:25.183 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-04-25T20:07:25.183 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2024-04-25T20:07:25.183 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-04-25T20:07:25.245 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-mgr-dashboard 2024-04-25T20:07:25.395 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-immutable-object-cache 2024-04-25T20:07:25.781 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:00:58 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:07:25.906 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-04-25T20:07:25.906 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:07:25.906 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-04-25T20:07:25.907 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:07:25.907 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-04-25T20:07:25.907 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1237.g646b8c7d.el8 ceph-noarch 3.5 M 2024-04-25T20:07:25.907 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2024-04-25T20:07:25.907 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1237.g646b8c7d.el8 ceph-noarch 24 k 2024-04-25T20:07:25.907 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1237.g646b8c7d.el8 ceph-noarch 15 k 2024-04-25T20:07:25.907 INFO:teuthology.orchestra.run.smithi039.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T20:07:25.907 INFO:teuthology.orchestra.run.smithi039.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T20:07:25.907 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:25.907 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-04-25T20:07:25.907 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:07:25.907 INFO:teuthology.orchestra.run.smithi039.stdout:Install 5 Packages 2024-04-25T20:07:25.907 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:25.908 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 3.8 M 2024-04-25T20:07:25.908 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 87 M 2024-04-25T20:07:25.908 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-04-25T20:07:25.930 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:00:58 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:07:26.051 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-25T20:07:26.052 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:07:26.052 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repo Size 2024-04-25T20:07:26.052 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:07:26.052 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-04-25T20:07:26.052 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 143 k 2024-04-25T20:07:26.052 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:26.053 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-04-25T20:07:26.053 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:07:26.053 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2024-04-25T20:07:26.053 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:26.053 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 143 k 2024-04-25T20:07:26.053 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 395 k 2024-04-25T20:07:26.053 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-04-25T20:07:26.310 INFO:teuthology.orchestra.run.smithi135.stdout:ceph-immutable-object-cache-18.2.2-1237.g646b8c 555 kB/s | 143 kB 00:00 2024-04-25T20:07:26.311 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:07:26.311 INFO:teuthology.orchestra.run.smithi135.stdout:Total 551 kB/s | 143 kB 00:00 2024-04-25T20:07:26.311 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-04-25T20:07:26.318 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-04-25T20:07:26.319 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-04-25T20:07:26.330 INFO:teuthology.orchestra.run.smithi039.stdout:(1/5): ceph-prometheus-alerts-18.2.2-1237.g646b 63 kB/s | 15 kB 00:00 2024-04-25T20:07:26.376 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-04-25T20:07:26.376 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-04-25T20:07:26.380 INFO:teuthology.orchestra.run.smithi039.stdout:(2/5): ceph-grafana-dashboards-18.2.2-1237.g646 83 kB/s | 24 kB 00:00 2024-04-25T20:07:26.519 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-04-25T20:07:26.572 INFO:teuthology.orchestra.run.smithi039.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 127 kB/s | 31 kB 00:00 2024-04-25T20:07:26.603 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1237.g646b8c7d. 1/1 2024-04-25T20:07:26.627 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1237.g646b8c7d. 1/1 2024-04-25T20:07:26.627 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:07:26.627 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T20:07:26.627 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:26.764 INFO:teuthology.orchestra.run.smithi039.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 492 kB/s | 188 kB 00:00 2024-04-25T20:07:27.223 INFO:teuthology.orchestra.run.smithi039.stdout:(5/5): ceph-mgr-dashboard-18.2.2-1237.g646b8c7d 3.1 MB/s | 3.5 MB 00:01 2024-04-25T20:07:27.225 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:07:27.225 INFO:teuthology.orchestra.run.smithi039.stdout:Total 2.9 MB/s | 3.8 MB 00:01 2024-04-25T20:07:27.225 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-04-25T20:07:27.234 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-04-25T20:07:27.234 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-04-25T20:07:27.302 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-04-25T20:07:27.302 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-04-25T20:07:27.501 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1237.g646b8c7d. 1/1 2024-04-25T20:07:27.501 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:27.501 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-04-25T20:07:27.501 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-immutable-object-cache-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:07:27.502 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:27.502 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-25T20:07:27.559 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-04-25T20:07:27.648 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-04-25T20:07:27.678 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-04-25T20:07:27.696 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-mgr 2024-04-25T20:07:27.707 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1237.g646b8c7d.el8.n 3/5 2024-04-25T20:07:28.216 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:01 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:07:28.297 INFO:teuthology.orchestra.run.smithi135.stdout:Package ceph-mgr-2:18.2.2-1237.g646b8c7d.el8.x86_64 is already installed. 2024-04-25T20:07:28.338 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-25T20:07:28.339 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2024-04-25T20:07:28.339 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-25T20:07:28.427 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-mgr-dashboard 2024-04-25T20:07:28.816 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1237.g646b8c7d.el8. 4/5 2024-04-25T20:07:28.860 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1237.g646b8c7d.el8.noarc 5/5 2024-04-25T20:07:28.942 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:01 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:07:29.066 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-25T20:07:29.067 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:07:29.067 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-04-25T20:07:29.067 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:07:29.067 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-04-25T20:07:29.067 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1237.g646b8c7d.el8 ceph-noarch 3.5 M 2024-04-25T20:07:29.067 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-04-25T20:07:29.067 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1237.g646b8c7d.el8 ceph-noarch 24 k 2024-04-25T20:07:29.067 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1237.g646b8c7d.el8 ceph-noarch 15 k 2024-04-25T20:07:29.067 INFO:teuthology.orchestra.run.smithi135.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T20:07:29.067 INFO:teuthology.orchestra.run.smithi135.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T20:07:29.067 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:29.068 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-04-25T20:07:29.068 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:07:29.068 INFO:teuthology.orchestra.run.smithi135.stdout:Install 5 Packages 2024-04-25T20:07:29.068 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:29.068 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 3.8 M 2024-04-25T20:07:29.068 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 87 M 2024-04-25T20:07:29.068 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-04-25T20:07:29.701 INFO:teuthology.orchestra.run.smithi135.stdout:(1/5): ceph-grafana-dashboards-18.2.2-1237.g646 98 kB/s | 24 kB 00:00 2024-04-25T20:07:29.734 INFO:teuthology.orchestra.run.smithi135.stdout:(2/5): ceph-prometheus-alerts-18.2.2-1237.g646b 54 kB/s | 15 kB 00:00 2024-04-25T20:07:29.943 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1237.g646b8c7d.el8.noarc 5/5 2024-04-25T20:07:29.943 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1237.g646b8c7d.el8. 1/5 2024-04-25T20:07:29.943 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1237.g646b8c7d.el8.noarc 2/5 2024-04-25T20:07:29.943 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1237.g646b8c7d.el8.n 3/5 2024-04-25T20:07:29.943 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-04-25T20:07:30.134 INFO:teuthology.orchestra.run.smithi135.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 71 kB/s | 31 kB 00:00 2024-04-25T20:07:30.192 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-04-25T20:07:30.192 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:30.192 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-04-25T20:07:30.192 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-grafana-dashboards-2:18.2.2-1237.g646b8c7d.el8.noarch 2024-04-25T20:07:30.192 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-dashboard-2:18.2.2-1237.g646b8c7d.el8.noarch 2024-04-25T20:07:30.192 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-prometheus-alerts-2:18.2.2-1237.g646b8c7d.el8.noarch 2024-04-25T20:07:30.193 INFO:teuthology.orchestra.run.smithi039.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T20:07:30.193 INFO:teuthology.orchestra.run.smithi039.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T20:07:30.193 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:30.193 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-04-25T20:07:30.360 INFO:teuthology.orchestra.run.smithi135.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 301 kB/s | 188 kB 00:00 2024-04-25T20:07:30.379 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-25T20:07:30.660 INFO:teuthology.orchestra.run.smithi135.stdout:(5/5): ceph-mgr-dashboard-18.2.2-1237.g646b8c7d 2.9 MB/s | 3.5 MB 00:01 2024-04-25T20:07:30.663 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:07:30.663 INFO:teuthology.orchestra.run.smithi135.stdout:Total 2.4 MB/s | 3.8 MB 00:01 2024-04-25T20:07:30.663 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-04-25T20:07:30.671 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-04-25T20:07:30.671 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-04-25T20:07:30.732 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-04-25T20:07:30.732 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-04-25T20:07:30.915 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:03 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:07:30.984 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-04-25T20:07:31.042 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-04-25T20:07:31.043 INFO:teuthology.orchestra.run.smithi039.stdout:======================================================================================= 2024-04-25T20:07:31.043 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-04-25T20:07:31.043 INFO:teuthology.orchestra.run.smithi039.stdout:======================================================================================= 2024-04-25T20:07:31.043 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-04-25T20:07:31.043 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1237.g646b8c7d.el8 ceph-noarch 7.4 M 2024-04-25T20:07:31.044 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2024-04-25T20:07:31.044 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T20:07:31.044 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T20:07:31.044 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T20:07:31.044 INFO:teuthology.orchestra.run.smithi039.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T20:07:31.044 INFO:teuthology.orchestra.run.smithi039.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T20:07:31.044 INFO:teuthology.orchestra.run.smithi039.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T20:07:31.044 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T20:07:31.044 INFO:teuthology.orchestra.run.smithi039.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T20:07:31.044 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T20:07:31.044 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T20:07:31.044 INFO:teuthology.orchestra.run.smithi039.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T20:07:31.044 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:31.044 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-04-25T20:07:31.044 INFO:teuthology.orchestra.run.smithi039.stdout:======================================================================================= 2024-04-25T20:07:31.045 INFO:teuthology.orchestra.run.smithi039.stdout:Install 12 Packages 2024-04-25T20:07:31.045 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:31.045 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 38 M 2024-04-25T20:07:31.045 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 209 M 2024-04-25T20:07:31.045 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-04-25T20:07:31.065 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-04-25T20:07:31.107 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-04-25T20:07:31.135 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1237.g646b8c7d.el8.n 3/5 2024-04-25T20:07:31.454 INFO:teuthology.orchestra.run.smithi039.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-04-25T20:07:31.513 INFO:teuthology.orchestra.run.smithi039.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 3.4 MB/s | 803 kB 00:00 2024-04-25T20:07:31.538 INFO:teuthology.orchestra.run.smithi039.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 393 kB/s | 33 kB 00:00 2024-04-25T20:07:31.580 INFO:teuthology.orchestra.run.smithi039.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 450 kB/s | 18 kB 00:00 2024-04-25T20:07:31.647 INFO:teuthology.orchestra.run.smithi039.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 22 MB/s | 3.0 MB 00:00 2024-04-25T20:07:31.673 INFO:teuthology.orchestra.run.smithi039.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 391 kB/s | 35 kB 00:00 2024-04-25T20:07:31.715 INFO:teuthology.orchestra.run.smithi039.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 6.0 MB/s | 244 kB 00:00 2024-04-25T20:07:31.798 INFO:teuthology.orchestra.run.smithi039.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 31 MB/s | 4.7 MB 00:00 2024-04-25T20:07:31.890 INFO:teuthology.orchestra.run.smithi039.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 5.1 MB 00:00 2024-04-25T20:07:31.916 INFO:teuthology.orchestra.run.smithi039.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.1 MB/s | 250 kB 00:00 2024-04-25T20:07:32.181 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1237.g646b8c7d.el8. 4/5 2024-04-25T20:07:32.223 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1237.g646b8c7d.el8.noarc 5/5 2024-04-25T20:07:32.307 INFO:teuthology.orchestra.run.smithi039.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 39 MB/s | 16 MB 00:00 2024-04-25T20:07:32.408 INFO:teuthology.orchestra.run.smithi039.stdout:(12/12): ceph-mgr-diskprediction-local-18.2.2-1 6.6 MB/s | 7.4 MB 00:01 2024-04-25T20:07:32.410 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:07:32.410 INFO:teuthology.orchestra.run.smithi039.stdout:Total 28 MB/s | 38 MB 00:01 2024-04-25T20:07:32.771 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-04-25T20:07:32.799 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-04-25T20:07:32.799 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-04-25T20:07:33.233 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-04-25T20:07:33.234 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-04-25T20:07:33.467 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1237.g646b8c7d.el8.noarc 5/5 2024-04-25T20:07:33.467 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1237.g646b8c7d.el8. 1/5 2024-04-25T20:07:33.467 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1237.g646b8c7d.el8.noarc 2/5 2024-04-25T20:07:33.467 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1237.g646b8c7d.el8.n 3/5 2024-04-25T20:07:33.467 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-04-25T20:07:33.716 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-04-25T20:07:33.716 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:33.716 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-04-25T20:07:33.716 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-grafana-dashboards-2:18.2.2-1237.g646b8c7d.el8.noarch 2024-04-25T20:07:33.716 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-dashboard-2:18.2.2-1237.g646b8c7d.el8.noarch 2024-04-25T20:07:33.716 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-prometheus-alerts-2:18.2.2-1237.g646b8c7d.el8.noarch 2024-04-25T20:07:33.716 INFO:teuthology.orchestra.run.smithi135.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T20:07:33.716 INFO:teuthology.orchestra.run.smithi135.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T20:07:33.716 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:33.716 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-25T20:07:33.921 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-25T20:07:34.039 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-04-25T20:07:34.104 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-25T20:07:34.148 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-25T20:07:34.251 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-25T20:07:34.290 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-25T20:07:34.435 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:07 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:07:34.474 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-25T20:07:34.506 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-25T20:07:34.557 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-25T20:07:34.558 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================= 2024-04-25T20:07:34.558 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-04-25T20:07:34.558 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================= 2024-04-25T20:07:34.558 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-04-25T20:07:34.558 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1237.g646b8c7d.el8 ceph-noarch 7.4 M 2024-04-25T20:07:34.558 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-04-25T20:07:34.558 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T20:07:34.558 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T20:07:34.558 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T20:07:34.558 INFO:teuthology.orchestra.run.smithi135.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T20:07:34.558 INFO:teuthology.orchestra.run.smithi135.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T20:07:34.558 INFO:teuthology.orchestra.run.smithi135.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T20:07:34.559 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T20:07:34.559 INFO:teuthology.orchestra.run.smithi135.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T20:07:34.559 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T20:07:34.559 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T20:07:34.559 INFO:teuthology.orchestra.run.smithi135.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T20:07:34.559 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:34.559 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-04-25T20:07:34.559 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================= 2024-04-25T20:07:34.559 INFO:teuthology.orchestra.run.smithi135.stdout:Install 12 Packages 2024-04-25T20:07:34.559 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:34.560 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 38 M 2024-04-25T20:07:34.560 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 209 M 2024-04-25T20:07:34.560 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-04-25T20:07:34.584 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-25T20:07:35.270 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-25T20:07:35.389 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-25T20:07:35.402 INFO:teuthology.orchestra.run.smithi135.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 583 kB/s | 195 kB 00:00 2024-04-25T20:07:35.494 INFO:teuthology.orchestra.run.smithi135.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 803 kB 00:00 2024-04-25T20:07:35.520 INFO:teuthology.orchestra.run.smithi135.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 279 kB/s | 33 kB 00:00 2024-04-25T20:07:35.645 INFO:teuthology.orchestra.run.smithi135.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 145 kB/s | 18 kB 00:00 2024-04-25T20:07:35.712 INFO:teuthology.orchestra.run.smithi135.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 14 MB/s | 3.0 MB 00:00 2024-04-25T20:07:35.738 INFO:teuthology.orchestra.run.smithi135.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 387 kB/s | 35 kB 00:00 2024-04-25T20:07:35.846 INFO:teuthology.orchestra.run.smithi135.stdout:(7/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 35 MB/s | 4.7 MB 00:00 2024-04-25T20:07:35.880 INFO:teuthology.orchestra.run.smithi135.stdout:(8/12): python3-devel-3.9.18-3.el9.x86_64.rpm 1.7 MB/s | 244 kB 00:00 2024-04-25T20:07:35.972 INFO:teuthology.orchestra.run.smithi135.stdout:(9/12): ceph-mgr-diskprediction-local-18.2.2-12 8.2 MB/s | 7.4 MB 00:00 2024-04-25T20:07:36.039 INFO:teuthology.orchestra.run.smithi135.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 5.1 MB 00:00 2024-04-25T20:07:36.064 INFO:teuthology.orchestra.run.smithi135.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.3 MB/s | 250 kB 00:00 2024-04-25T20:07:36.431 INFO:teuthology.orchestra.run.smithi135.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 36 MB/s | 16 MB 00:00 2024-04-25T20:07:36.434 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:07:36.434 INFO:teuthology.orchestra.run.smithi135.stdout:Total 20 MB/s | 38 MB 00:01 2024-04-25T20:07:36.697 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-25T20:07:36.785 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-04-25T20:07:36.812 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-04-25T20:07:36.812 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-04-25T20:07:37.222 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-04-25T20:07:37.222 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-04-25T20:07:37.792 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-25T20:07:37.821 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1237.g646b8 12/12 2024-04-25T20:07:37.978 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-04-25T20:07:38.069 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-25T20:07:38.113 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-25T20:07:38.214 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-25T20:07:38.256 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-25T20:07:38.433 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-25T20:07:38.472 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-25T20:07:38.549 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-25T20:07:39.212 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-25T20:07:39.329 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-25T20:07:40.270 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1237.g646b8 12/12 2024-04-25T20:07:40.271 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1237.g646b8 1/12 2024-04-25T20:07:40.271 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-25T20:07:40.271 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-25T20:07:40.271 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-25T20:07:40.271 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-25T20:07:40.271 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-25T20:07:40.271 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-25T20:07:40.271 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-25T20:07:40.271 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-25T20:07:40.271 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-25T20:07:40.271 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-25T20:07:40.567 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-25T20:07:40.635 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-25T20:07:40.635 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:40.635 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-04-25T20:07:40.635 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1237.g646b8c7d.el8.noarch 2024-04-25T20:07:40.635 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T20:07:40.636 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T20:07:40.636 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T20:07:40.636 INFO:teuthology.orchestra.run.smithi039.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T20:07:40.636 INFO:teuthology.orchestra.run.smithi039.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T20:07:40.636 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T20:07:40.636 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T20:07:40.636 INFO:teuthology.orchestra.run.smithi039.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T20:07:40.636 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T20:07:40.636 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T20:07:40.636 INFO:teuthology.orchestra.run.smithi039.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T20:07:40.636 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:40.636 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-04-25T20:07:41.016 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-mgr-rook 2024-04-25T20:07:41.565 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:14 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:07:41.619 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-25T20:07:41.643 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1237.g646b8 12/12 2024-04-25T20:07:41.691 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-04-25T20:07:41.692 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:07:41.692 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-04-25T20:07:41.693 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:07:41.693 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-04-25T20:07:41.693 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-rook noarch 2:18.2.2-1237.g646b8c7d.el8 ceph-noarch 49 k 2024-04-25T20:07:41.693 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2024-04-25T20:07:41.693 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T20:07:41.693 INFO:teuthology.orchestra.run.smithi039.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T20:07:41.693 INFO:teuthology.orchestra.run.smithi039.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T20:07:41.693 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T20:07:41.693 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T20:07:41.693 INFO:teuthology.orchestra.run.smithi039.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T20:07:41.693 INFO:teuthology.orchestra.run.smithi039.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T20:07:41.694 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T20:07:41.694 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T20:07:41.694 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T20:07:41.694 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T20:07:41.694 INFO:teuthology.orchestra.run.smithi039.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T20:07:41.694 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:41.694 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-04-25T20:07:41.694 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:07:41.694 INFO:teuthology.orchestra.run.smithi039.stdout:Install 13 Packages 2024-04-25T20:07:41.694 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:41.695 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 2.2 M 2024-04-25T20:07:41.695 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 26 M 2024-04-25T20:07:41.695 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-04-25T20:07:42.247 INFO:teuthology.orchestra.run.smithi039.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 204 kB/s | 26 kB 00:00 2024-04-25T20:07:42.280 INFO:teuthology.orchestra.run.smithi039.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 120 kB/s | 19 kB 00:00 2024-04-25T20:07:42.330 INFO:teuthology.orchestra.run.smithi039.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.6 MB/s | 222 kB 00:00 2024-04-25T20:07:42.355 INFO:teuthology.orchestra.run.smithi039.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-25T20:07:42.381 INFO:teuthology.orchestra.run.smithi039.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 5.4 MB/s | 279 kB 00:00 2024-04-25T20:07:42.406 INFO:teuthology.orchestra.run.smithi039.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-04-25T20:07:42.515 INFO:teuthology.orchestra.run.smithi039.stdout:(7/13): python3-certifi-2021.10.8-2.el9.noarch. 138 kB/s | 15 kB 00:00 2024-04-25T20:07:42.540 INFO:teuthology.orchestra.run.smithi039.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 202 kB/s | 32 kB 00:00 2024-04-25T20:07:42.582 INFO:teuthology.orchestra.run.smithi039.stdout:(9/13): ceph-mgr-rook-18.2.2-1237.g646b8c7d.el8 107 kB/s | 49 kB 00:00 2024-04-25T20:07:42.682 INFO:teuthology.orchestra.run.smithi039.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 1.2 MB/s | 206 kB 00:00 2024-04-25T20:07:42.733 INFO:teuthology.orchestra.run.smithi039.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 393 kB/s | 59 kB 00:00 2024-04-25T20:07:42.758 INFO:teuthology.orchestra.run.smithi039.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.2 MB/s | 90 kB 00:00 2024-04-25T20:07:42.791 INFO:teuthology.orchestra.run.smithi039.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 4.1 MB/s | 1.0 MB 00:00 2024-04-25T20:07:42.793 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:07:42.793 INFO:teuthology.orchestra.run.smithi039.stdout:Total 2.0 MB/s | 2.2 MB 00:01 2024-04-25T20:07:42.858 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-04-25T20:07:42.875 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-04-25T20:07:42.875 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-04-25T20:07:42.969 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-04-25T20:07:42.970 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-04-25T20:07:43.545 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-04-25T20:07:43.725 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-25T20:07:43.787 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-25T20:07:43.842 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-25T20:07:43.879 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-25T20:07:43.920 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-25T20:07:44.018 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-25T20:07:44.146 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-25T20:07:44.197 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-25T20:07:44.299 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1237.g646b8 12/12 2024-04-25T20:07:44.300 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1237.g646b8 1/12 2024-04-25T20:07:44.300 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-25T20:07:44.300 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-25T20:07:44.300 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-25T20:07:44.300 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-25T20:07:44.300 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-25T20:07:44.300 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-25T20:07:44.300 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-25T20:07:44.300 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-25T20:07:44.300 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-25T20:07:44.300 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-25T20:07:44.673 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-25T20:07:44.673 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:44.673 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-04-25T20:07:44.673 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1237.g646b8c7d.el8.noarch 2024-04-25T20:07:44.673 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T20:07:44.673 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T20:07:44.673 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T20:07:44.673 INFO:teuthology.orchestra.run.smithi135.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T20:07:44.673 INFO:teuthology.orchestra.run.smithi135.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T20:07:44.673 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T20:07:44.674 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T20:07:44.674 INFO:teuthology.orchestra.run.smithi135.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T20:07:44.674 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T20:07:44.674 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T20:07:44.674 INFO:teuthology.orchestra.run.smithi135.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T20:07:44.674 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:44.674 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-25T20:07:44.978 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-25T20:07:45.026 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-25T20:07:45.059 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-25T20:07:45.066 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-mgr-rook 2024-04-25T20:07:45.109 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-25T20:07:45.137 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-rook-2:18.2.2-1237.g646b8c7d.el8.noarch 13/13 2024-04-25T20:07:45.589 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:18 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:07:45.716 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-25T20:07:45.717 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:07:45.717 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-04-25T20:07:45.718 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:07:45.718 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-04-25T20:07:45.718 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-rook noarch 2:18.2.2-1237.g646b8c7d.el8 ceph-noarch 49 k 2024-04-25T20:07:45.718 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-04-25T20:07:45.718 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T20:07:45.718 INFO:teuthology.orchestra.run.smithi135.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T20:07:45.718 INFO:teuthology.orchestra.run.smithi135.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T20:07:45.718 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T20:07:45.718 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T20:07:45.718 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T20:07:45.718 INFO:teuthology.orchestra.run.smithi135.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T20:07:45.718 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T20:07:45.719 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T20:07:45.719 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T20:07:45.719 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T20:07:45.719 INFO:teuthology.orchestra.run.smithi135.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T20:07:45.719 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:45.719 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-04-25T20:07:45.719 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:07:45.719 INFO:teuthology.orchestra.run.smithi135.stdout:Install 13 Packages 2024-04-25T20:07:45.719 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:45.720 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 2.2 M 2024-04-25T20:07:45.720 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 26 M 2024-04-25T20:07:45.720 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-04-25T20:07:45.877 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1237.g646b8c7d.el8.noarch 13/13 2024-04-25T20:07:45.877 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1237.g646b8c7d.el8.noarch 1/13 2024-04-25T20:07:45.878 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-25T20:07:45.878 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-25T20:07:45.878 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-25T20:07:45.878 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-25T20:07:45.878 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-25T20:07:45.878 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-25T20:07:45.879 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-25T20:07:45.879 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-25T20:07:45.879 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-25T20:07:45.879 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-25T20:07:45.879 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-25T20:07:46.182 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-25T20:07:46.182 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:46.182 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-04-25T20:07:46.182 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-rook-2:18.2.2-1237.g646b8c7d.el8.noarch 2024-04-25T20:07:46.182 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T20:07:46.182 INFO:teuthology.orchestra.run.smithi039.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T20:07:46.182 INFO:teuthology.orchestra.run.smithi039.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T20:07:46.182 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T20:07:46.182 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T20:07:46.182 INFO:teuthology.orchestra.run.smithi039.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T20:07:46.183 INFO:teuthology.orchestra.run.smithi039.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T20:07:46.183 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T20:07:46.183 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T20:07:46.183 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T20:07:46.183 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T20:07:46.183 INFO:teuthology.orchestra.run.smithi039.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T20:07:46.183 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:46.183 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-04-25T20:07:46.231 INFO:teuthology.orchestra.run.smithi135.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 100 kB/s | 19 kB 00:00 2024-04-25T20:07:46.249 INFO:teuthology.orchestra.run.smithi135.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 123 kB/s | 26 kB 00:00 2024-04-25T20:07:46.341 INFO:teuthology.orchestra.run.smithi135.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-04-25T20:07:46.374 INFO:teuthology.orchestra.run.smithi135.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-04-25T20:07:46.397 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-mgr-cephadm 2024-04-25T20:07:46.408 INFO:teuthology.orchestra.run.smithi135.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 279 kB 00:00 2024-04-25T20:07:46.433 INFO:teuthology.orchestra.run.smithi135.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 919 kB/s | 54 kB 00:00 2024-04-25T20:07:46.459 INFO:teuthology.orchestra.run.smithi135.stdout:(7/13): ceph-mgr-rook-18.2.2-1237.g646b8c7d.el8 118 kB/s | 49 kB 00:00 2024-04-25T20:07:46.726 INFO:teuthology.orchestra.run.smithi135.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 51 kB/s | 15 kB 00:00 2024-04-25T20:07:46.760 INFO:teuthology.orchestra.run.smithi135.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 92 kB/s | 32 kB 00:00 2024-04-25T20:07:46.893 INFO:teuthology.orchestra.run.smithi135.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 473 kB/s | 206 kB 00:00 2024-04-25T20:07:46.919 INFO:teuthology.orchestra.run.smithi135.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 372 kB/s | 59 kB 00:00 2024-04-25T20:07:46.940 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:19 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:07:46.985 INFO:teuthology.orchestra.run.smithi135.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 978 kB/s | 90 kB 00:00 2024-04-25T20:07:47.066 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-04-25T20:07:47.070 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:07:47.070 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-04-25T20:07:47.070 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:07:47.070 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-04-25T20:07:47.070 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1237.g646b8c7d.el8 ceph-noarch 138 k 2024-04-25T20:07:47.070 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2024-04-25T20:07:47.071 INFO:teuthology.orchestra.run.smithi039.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T20:07:47.071 INFO:teuthology.orchestra.run.smithi039.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T20:07:47.071 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T20:07:47.071 INFO:teuthology.orchestra.run.smithi039.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T20:07:47.071 INFO:teuthology.orchestra.run.smithi039.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T20:07:47.071 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:47.071 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-04-25T20:07:47.071 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:07:47.071 INFO:teuthology.orchestra.run.smithi039.stdout:Install 6 Packages 2024-04-25T20:07:47.071 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:47.071 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 7.0 M 2024-04-25T20:07:47.071 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 33 M 2024-04-25T20:07:47.071 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-04-25T20:07:47.128 INFO:teuthology.orchestra.run.smithi135.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.5 MB/s | 1.0 MB 00:00 2024-04-25T20:07:47.132 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:07:47.132 INFO:teuthology.orchestra.run.smithi135.stdout:Total 1.6 MB/s | 2.2 MB 00:01 2024-04-25T20:07:47.194 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-04-25T20:07:47.209 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-04-25T20:07:47.209 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-04-25T20:07:47.294 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-04-25T20:07:47.295 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-04-25T20:07:47.477 INFO:teuthology.orchestra.run.smithi039.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.4 MB/s | 248 kB 00:00 2024-04-25T20:07:47.660 INFO:teuthology.orchestra.run.smithi039.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-04-25T20:07:47.685 INFO:teuthology.orchestra.run.smithi039.stdout:(3/6): ceph-mgr-cephadm-18.2.2-1237.g646b8c7d.e 358 kB/s | 138 kB 00:00 2024-04-25T20:07:47.880 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-04-25T20:07:47.969 INFO:teuthology.orchestra.run.smithi039.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 188 kB/s | 58 kB 00:00 2024-04-25T20:07:48.002 INFO:teuthology.orchestra.run.smithi039.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 161 kB/s | 51 kB 00:00 2024-04-25T20:07:48.028 INFO:teuthology.orchestra.run.smithi039.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 997 kB/s | 548 kB 00:00 2024-04-25T20:07:48.032 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:07:48.032 INFO:teuthology.orchestra.run.smithi039.stdout:Total 7.2 MB/s | 7.0 MB 00:00 2024-04-25T20:07:48.062 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-25T20:07:48.104 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-04-25T20:07:48.117 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-04-25T20:07:48.117 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-04-25T20:07:48.121 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-25T20:07:48.177 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-25T20:07:48.215 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-25T20:07:48.221 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-04-25T20:07:48.223 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-04-25T20:07:48.256 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-25T20:07:48.351 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-25T20:07:48.460 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-25T20:07:48.511 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-25T20:07:48.527 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-04-25T20:07:48.709 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-25T20:07:48.772 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-25T20:07:49.236 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-25T20:07:49.291 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-25T20:07:49.325 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-25T20:07:49.328 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-25T20:07:49.372 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-25T20:07:49.394 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-rook-2:18.2.2-1237.g646b8c7d.el8.noarch 13/13 2024-04-25T20:07:49.419 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-25T20:07:49.489 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-25T20:07:49.517 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1237.g646b8c7d.el8.noarch 6/6 2024-04-25T20:07:50.250 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1237.g646b8c7d.el8.noarch 6/6 2024-04-25T20:07:50.250 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1237.g646b8c7d.el8.noarch 1/6 2024-04-25T20:07:50.250 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-25T20:07:50.250 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-25T20:07:50.250 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-25T20:07:50.250 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-25T20:07:50.280 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1237.g646b8c7d.el8.noarch 13/13 2024-04-25T20:07:50.280 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1237.g646b8c7d.el8.noarch 1/13 2024-04-25T20:07:50.280 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-25T20:07:50.280 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-25T20:07:50.280 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-25T20:07:50.280 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-25T20:07:50.280 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-25T20:07:50.281 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-25T20:07:50.281 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-25T20:07:50.281 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-25T20:07:50.281 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-25T20:07:50.281 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-25T20:07:50.281 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-25T20:07:50.514 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-25T20:07:50.514 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:50.514 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-04-25T20:07:50.514 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-cephadm-2:18.2.2-1237.g646b8c7d.el8.noarch 2024-04-25T20:07:50.514 INFO:teuthology.orchestra.run.smithi039.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T20:07:50.514 INFO:teuthology.orchestra.run.smithi039.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T20:07:50.514 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T20:07:50.515 INFO:teuthology.orchestra.run.smithi039.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T20:07:50.515 INFO:teuthology.orchestra.run.smithi039.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T20:07:50.515 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:50.515 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-04-25T20:07:50.580 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-25T20:07:50.580 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:50.580 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-04-25T20:07:50.580 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-rook-2:18.2.2-1237.g646b8c7d.el8.noarch 2024-04-25T20:07:50.580 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T20:07:50.580 INFO:teuthology.orchestra.run.smithi135.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T20:07:50.580 INFO:teuthology.orchestra.run.smithi135.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T20:07:50.580 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T20:07:50.580 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T20:07:50.580 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T20:07:50.580 INFO:teuthology.orchestra.run.smithi135.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T20:07:50.580 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T20:07:50.581 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T20:07:50.581 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T20:07:50.581 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T20:07:50.581 INFO:teuthology.orchestra.run.smithi135.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T20:07:50.581 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:50.581 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-25T20:07:50.733 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-fuse 2024-04-25T20:07:50.828 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-mgr-cephadm 2024-04-25T20:07:51.270 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:24 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:07:51.342 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:24 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:07:51.397 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-04-25T20:07:51.398 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:07:51.398 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-04-25T20:07:51.398 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:07:51.398 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-04-25T20:07:51.398 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-fuse x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 848 k 2024-04-25T20:07:51.398 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2024-04-25T20:07:51.398 INFO:teuthology.orchestra.run.smithi039.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T20:07:51.399 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:51.399 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-04-25T20:07:51.399 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:07:51.399 INFO:teuthology.orchestra.run.smithi039.stdout:Install 2 Packages 2024-04-25T20:07:51.399 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:51.399 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 928 k 2024-04-25T20:07:51.399 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 2.7 M 2024-04-25T20:07:51.399 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-04-25T20:07:51.463 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-25T20:07:51.464 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:07:51.464 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-04-25T20:07:51.464 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:07:51.464 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-04-25T20:07:51.464 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1237.g646b8c7d.el8 ceph-noarch 138 k 2024-04-25T20:07:51.464 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-04-25T20:07:51.465 INFO:teuthology.orchestra.run.smithi135.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T20:07:51.465 INFO:teuthology.orchestra.run.smithi135.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T20:07:51.465 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T20:07:51.465 INFO:teuthology.orchestra.run.smithi135.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T20:07:51.465 INFO:teuthology.orchestra.run.smithi135.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T20:07:51.465 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:51.465 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-04-25T20:07:51.465 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:07:51.465 INFO:teuthology.orchestra.run.smithi135.stdout:Install 6 Packages 2024-04-25T20:07:51.465 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:51.466 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 7.0 M 2024-04-25T20:07:51.466 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 33 M 2024-04-25T20:07:51.466 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-04-25T20:07:51.738 INFO:teuthology.orchestra.run.smithi039.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 513 kB/s | 80 kB 00:00 2024-04-25T20:07:52.150 INFO:teuthology.orchestra.run.smithi135.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 838 kB/s | 248 kB 00:00 2024-04-25T20:07:52.317 INFO:teuthology.orchestra.run.smithi135.stdout:(2/6): ceph-mgr-cephadm-18.2.2-1237.g646b8c7d.e 299 kB/s | 138 kB 00:00 2024-04-25T20:07:52.371 INFO:teuthology.orchestra.run.smithi039.stdout:(2/2): ceph-fuse-18.2.2-1237.g646b8c7d.el8.x86_ 1.0 MB/s | 848 kB 00:00 2024-04-25T20:07:52.372 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:07:52.373 INFO:teuthology.orchestra.run.smithi039.stdout:Total 953 kB/s | 928 kB 00:00 2024-04-25T20:07:52.385 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-04-25T20:07:52.392 INFO:teuthology.orchestra.run.smithi135.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-04-25T20:07:52.408 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-04-25T20:07:52.408 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-04-25T20:07:52.489 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-04-25T20:07:52.489 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-04-25T20:07:52.492 INFO:teuthology.orchestra.run.smithi135.stdout:(4/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.6 MB/s | 548 kB 00:00 2024-04-25T20:07:52.525 INFO:teuthology.orchestra.run.smithi135.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 277 kB/s | 58 kB 00:00 2024-04-25T20:07:52.542 INFO:teuthology.orchestra.run.smithi135.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 339 kB/s | 51 kB 00:00 2024-04-25T20:07:52.544 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:07:52.545 INFO:teuthology.orchestra.run.smithi135.stdout:Total 6.5 MB/s | 7.0 MB 00:01 2024-04-25T20:07:52.615 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-04-25T20:07:52.625 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-04-25T20:07:52.625 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-04-25T20:07:52.676 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-04-25T20:07:52.722 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-04-25T20:07:52.722 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-04-25T20:07:52.807 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-25T20:07:52.874 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-fuse-2:18.2.2-1237.g646b8c7d.el8.x86_64 2/2 2024-04-25T20:07:53.024 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-04-25T20:07:53.198 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-25T20:07:53.252 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-25T20:07:53.506 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-fuse-2:18.2.2-1237.g646b8c7d.el8.x86_64 2/2 2024-04-25T20:07:53.507 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-fuse-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/2 2024-04-25T20:07:53.757 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-25T20:07:53.757 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:53.757 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-04-25T20:07:53.757 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-fuse-2:18.2.2-1237.g646b8c7d.el8.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-25T20:07:53.757 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:53.757 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-04-25T20:07:53.769 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-25T20:07:53.863 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-25T20:07:53.925 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-25T20:07:53.948 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1237.g646b8c7d.el8.noarch 6/6 2024-04-25T20:07:53.966 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-volume 2024-04-25T20:07:54.507 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:27 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:07:54.632 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-04-25T20:07:54.632 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:07:54.632 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-04-25T20:07:54.633 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:07:54.633 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-04-25T20:07:54.633 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-volume noarch 2:18.2.2-1237.g646b8c7d.el8 ceph-noarch 264 k 2024-04-25T20:07:54.633 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2024-04-25T20:07:54.633 INFO:teuthology.orchestra.run.smithi039.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T20:07:54.633 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:54.633 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-04-25T20:07:54.633 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:07:54.633 INFO:teuthology.orchestra.run.smithi039.stdout:Install 2 Packages 2024-04-25T20:07:54.633 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:54.634 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 341 k 2024-04-25T20:07:54.634 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 1.5 M 2024-04-25T20:07:54.634 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-04-25T20:07:54.668 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1237.g646b8c7d.el8.noarch 6/6 2024-04-25T20:07:54.668 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1237.g646b8c7d.el8.noarch 1/6 2024-04-25T20:07:54.668 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-25T20:07:54.669 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-25T20:07:54.669 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-25T20:07:54.669 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-25T20:07:54.887 INFO:teuthology.orchestra.run.smithi039.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 573 kB/s | 77 kB 00:00 2024-04-25T20:07:54.918 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-25T20:07:54.918 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:54.918 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-04-25T20:07:54.918 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-cephadm-2:18.2.2-1237.g646b8c7d.el8.noarch 2024-04-25T20:07:54.919 INFO:teuthology.orchestra.run.smithi135.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T20:07:54.919 INFO:teuthology.orchestra.run.smithi135.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T20:07:54.919 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T20:07:54.919 INFO:teuthology.orchestra.run.smithi135.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T20:07:54.919 INFO:teuthology.orchestra.run.smithi135.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T20:07:54.919 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:54.919 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-25T20:07:55.139 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-fuse 2024-04-25T20:07:55.171 INFO:teuthology.orchestra.run.smithi039.stdout:(2/2): ceph-volume-18.2.2-1237.g646b8c7d.el8.no 630 kB/s | 264 kB 00:00 2024-04-25T20:07:55.172 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:07:55.172 INFO:teuthology.orchestra.run.smithi039.stdout:Total 632 kB/s | 341 kB 00:00 2024-04-25T20:07:55.184 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-04-25T20:07:55.194 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-04-25T20:07:55.195 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-04-25T20:07:55.223 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-04-25T20:07:55.223 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-04-25T20:07:55.375 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-04-25T20:07:55.574 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-04-25T20:07:55.615 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-volume-2:18.2.2-1237.g646b8c7d.el8.noarch 2/2 2024-04-25T20:07:55.631 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-volume-2:18.2.2-1237.g646b8c7d.el8.noarch 2/2 2024-04-25T20:07:55.631 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:07:55.631 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T20:07:55.632 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:55.654 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:28 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:07:55.771 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-25T20:07:55.772 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:07:55.772 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-04-25T20:07:55.772 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:07:55.772 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-04-25T20:07:55.772 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-fuse x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 848 k 2024-04-25T20:07:55.772 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-04-25T20:07:55.772 INFO:teuthology.orchestra.run.smithi135.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T20:07:55.773 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:55.773 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-04-25T20:07:55.773 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:07:55.773 INFO:teuthology.orchestra.run.smithi135.stdout:Install 2 Packages 2024-04-25T20:07:55.773 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:55.773 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 928 k 2024-04-25T20:07:55.773 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 2.7 M 2024-04-25T20:07:55.773 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-04-25T20:07:56.194 INFO:teuthology.orchestra.run.smithi135.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 544 kB/s | 80 kB 00:00 2024-04-25T20:07:56.319 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-volume-2:18.2.2-1237.g646b8c7d.el8.noarch 1/2 2024-04-25T20:07:56.622 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-04-25T20:07:56.622 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:56.622 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-04-25T20:07:56.622 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-volume-2:18.2.2-1237.g646b8c7d.el8.noarch 2024-04-25T20:07:56.622 INFO:teuthology.orchestra.run.smithi039.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T20:07:56.622 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:56.622 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-04-25T20:07:56.627 INFO:teuthology.orchestra.run.smithi135.stdout:(2/2): ceph-fuse-18.2.2-1237.g646b8c7d.el8.x86_ 1.4 MB/s | 848 kB 00:00 2024-04-25T20:07:56.629 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:07:56.629 INFO:teuthology.orchestra.run.smithi135.stdout:Total 1.1 MB/s | 928 kB 00:00 2024-04-25T20:07:56.641 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-04-25T20:07:56.663 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-04-25T20:07:56.663 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-04-25T20:07:56.742 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-04-25T20:07:56.743 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-04-25T20:07:56.906 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install librados-devel 2024-04-25T20:07:56.929 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-04-25T20:07:57.075 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-25T20:07:57.150 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-fuse-2:18.2.2-1237.g646b8c7d.el8.x86_64 2/2 2024-04-25T20:07:57.444 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:30 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:07:57.569 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-04-25T20:07:57.570 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:07:57.570 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repo Size 2024-04-25T20:07:57.570 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:07:57.570 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-04-25T20:07:57.570 INFO:teuthology.orchestra.run.smithi039.stdout: librados-devel x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 127 k 2024-04-25T20:07:57.570 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:57.570 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-04-25T20:07:57.571 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:07:57.571 INFO:teuthology.orchestra.run.smithi039.stdout:Install 1 Package 2024-04-25T20:07:57.571 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:57.571 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 127 k 2024-04-25T20:07:57.571 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 456 k 2024-04-25T20:07:57.571 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-04-25T20:07:57.798 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-fuse-2:18.2.2-1237.g646b8c7d.el8.x86_64 2/2 2024-04-25T20:07:57.798 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-fuse-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/2 2024-04-25T20:07:57.803 INFO:teuthology.orchestra.run.smithi039.stdout:librados-devel-18.2.2-1237.g646b8c7d.el8.x86_64 548 kB/s | 127 kB 00:00 2024-04-25T20:07:57.803 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:07:57.804 INFO:teuthology.orchestra.run.smithi039.stdout:Total 544 kB/s | 127 kB 00:00 2024-04-25T20:07:57.804 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-04-25T20:07:57.809 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-04-25T20:07:57.810 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-04-25T20:07:57.875 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-04-25T20:07:57.876 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-04-25T20:07:58.028 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-04-25T20:07:58.042 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-25T20:07:58.042 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:58.043 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-04-25T20:07:58.043 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-fuse-2:18.2.2-1237.g646b8c7d.el8.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-25T20:07:58.043 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:58.043 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-25T20:07:58.119 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librados-devel-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:07:58.307 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-volume 2024-04-25T20:07:58.493 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librados-devel-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:07:58.729 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados-devel-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:07:58.729 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:58.729 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-04-25T20:07:58.729 INFO:teuthology.orchestra.run.smithi039.stdout: librados-devel-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:07:58.729 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:07:58.729 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-04-25T20:07:58.832 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:31 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:07:58.954 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-25T20:07:58.954 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:07:58.954 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-04-25T20:07:58.955 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:07:58.955 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-04-25T20:07:58.955 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-volume noarch 2:18.2.2-1237.g646b8c7d.el8 ceph-noarch 264 k 2024-04-25T20:07:58.955 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-04-25T20:07:58.955 INFO:teuthology.orchestra.run.smithi135.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T20:07:58.955 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:58.955 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-04-25T20:07:58.955 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:07:58.955 INFO:teuthology.orchestra.run.smithi135.stdout:Install 2 Packages 2024-04-25T20:07:58.955 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:07:58.956 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 341 k 2024-04-25T20:07:58.956 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 1.5 M 2024-04-25T20:07:58.956 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-04-25T20:07:58.964 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install libcephfs2 2024-04-25T20:07:59.348 INFO:teuthology.orchestra.run.smithi135.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 656 kB/s | 77 kB 00:00 2024-04-25T20:07:59.509 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:32 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:07:59.590 INFO:teuthology.orchestra.run.smithi135.stdout:(2/2): ceph-volume-18.2.2-1237.g646b8c7d.el8.no 733 kB/s | 264 kB 00:00 2024-04-25T20:07:59.591 INFO:teuthology.orchestra.run.smithi039.stdout:Package libcephfs2-2:18.2.2-1237.g646b8c7d.el8.x86_64 is already installed. 2024-04-25T20:07:59.591 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:07:59.592 INFO:teuthology.orchestra.run.smithi135.stdout:Total 536 kB/s | 341 kB 00:00 2024-04-25T20:07:59.603 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-04-25T20:07:59.614 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-04-25T20:07:59.614 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-04-25T20:07:59.634 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-04-25T20:07:59.635 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2024-04-25T20:07:59.635 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-04-25T20:07:59.639 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-04-25T20:07:59.640 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-04-25T20:07:59.698 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install libcephfs-devel 2024-04-25T20:07:59.798 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-04-25T20:07:59.990 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-04-25T20:08:00.042 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-volume-2:18.2.2-1237.g646b8c7d.el8.noarch 2/2 2024-04-25T20:08:00.057 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-volume-2:18.2.2-1237.g646b8c7d.el8.noarch 2/2 2024-04-25T20:08:00.057 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:08:00.057 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T20:08:00.057 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:00.229 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:33 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:08:00.353 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-04-25T20:08:00.354 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:08:00.354 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repo Size 2024-04-25T20:08:00.354 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:08:00.354 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-04-25T20:08:00.354 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs-devel x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 31 k 2024-04-25T20:08:00.355 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:00.355 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-04-25T20:08:00.355 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:08:00.355 INFO:teuthology.orchestra.run.smithi039.stdout:Install 1 Package 2024-04-25T20:08:00.355 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:00.355 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 31 k 2024-04-25T20:08:00.355 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 139 k 2024-04-25T20:08:00.355 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-04-25T20:08:00.726 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-volume-2:18.2.2-1237.g646b8c7d.el8.noarch 1/2 2024-04-25T20:08:00.957 INFO:teuthology.orchestra.run.smithi039.stdout:libcephfs-devel-18.2.2-1237.g646b8c7d.el8.x86_6 52 kB/s | 31 kB 00:00 2024-04-25T20:08:00.958 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:08:00.958 INFO:teuthology.orchestra.run.smithi039.stdout:Total 51 kB/s | 31 kB 00:00 2024-04-25T20:08:00.958 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-04-25T20:08:00.962 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-04-25T20:08:00.963 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-04-25T20:08:00.965 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-04-25T20:08:00.965 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-04-25T20:08:01.000 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-04-25T20:08:01.001 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:01.001 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-04-25T20:08:01.001 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-volume-2:18.2.2-1237.g646b8c7d.el8.noarch 2024-04-25T20:08:01.001 INFO:teuthology.orchestra.run.smithi135.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T20:08:01.001 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:01.001 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-25T20:08:01.054 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-04-25T20:08:01.180 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephfs-devel-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:08:01.330 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install librados-devel 2024-04-25T20:08:01.538 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libcephfs-devel-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:08:01.754 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephfs-devel-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:08:01.754 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:01.754 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-04-25T20:08:01.754 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs-devel-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:08:01.754 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:01.754 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-04-25T20:08:01.845 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:34 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:08:01.917 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install librados2 2024-04-25T20:08:01.966 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-25T20:08:01.967 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:08:01.967 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repo Size 2024-04-25T20:08:01.967 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:08:01.967 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-04-25T20:08:01.967 INFO:teuthology.orchestra.run.smithi135.stdout: librados-devel x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 127 k 2024-04-25T20:08:01.967 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:01.967 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-04-25T20:08:01.967 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:08:01.967 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2024-04-25T20:08:01.967 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:01.968 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 127 k 2024-04-25T20:08:01.968 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 456 k 2024-04-25T20:08:01.968 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-04-25T20:08:02.455 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:35 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:08:02.536 INFO:teuthology.orchestra.run.smithi039.stdout:Package librados2-2:18.2.2-1237.g646b8c7d.el8.x86_64 is already installed. 2024-04-25T20:08:02.578 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-04-25T20:08:02.580 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2024-04-25T20:08:02.581 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-04-25T20:08:02.642 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install librbd1 2024-04-25T20:08:02.704 INFO:teuthology.orchestra.run.smithi135.stdout:librados-devel-18.2.2-1237.g646b8c7d.el8.x86_64 173 kB/s | 127 kB 00:00 2024-04-25T20:08:02.704 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:08:02.704 INFO:teuthology.orchestra.run.smithi135.stdout:Total 172 kB/s | 127 kB 00:00 2024-04-25T20:08:02.705 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-04-25T20:08:02.710 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-04-25T20:08:02.710 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-04-25T20:08:02.774 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-04-25T20:08:02.774 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-04-25T20:08:02.922 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-04-25T20:08:03.109 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librados-devel-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:08:03.182 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:36 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:08:03.263 INFO:teuthology.orchestra.run.smithi039.stdout:Package librbd1-2:18.2.2-1237.g646b8c7d.el8.x86_64 is already installed. 2024-04-25T20:08:03.307 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-04-25T20:08:03.308 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2024-04-25T20:08:03.309 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-04-25T20:08:03.375 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install python3-rados 2024-04-25T20:08:03.482 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados-devel-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:08:03.741 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados-devel-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:08:03.742 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:03.742 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-04-25T20:08:03.742 INFO:teuthology.orchestra.run.smithi135.stdout: librados-devel-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:08:03.742 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:03.742 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-25T20:08:03.919 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:36 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:08:03.963 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install libcephfs2 2024-04-25T20:08:04.005 INFO:teuthology.orchestra.run.smithi039.stdout:Package python3-rados-2:18.2.2-1237.g646b8c7d.el8.x86_64 is already installed. 2024-04-25T20:08:04.049 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-04-25T20:08:04.050 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2024-04-25T20:08:04.050 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-04-25T20:08:04.108 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install python3-rgw 2024-04-25T20:08:04.479 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:37 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:08:04.560 INFO:teuthology.orchestra.run.smithi135.stdout:Package libcephfs2-2:18.2.2-1237.g646b8c7d.el8.x86_64 is already installed. 2024-04-25T20:08:04.602 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-25T20:08:04.603 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2024-04-25T20:08:04.603 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-25T20:08:04.651 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:37 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:08:04.667 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install libcephfs-devel 2024-04-25T20:08:04.733 INFO:teuthology.orchestra.run.smithi039.stdout:Package python3-rgw-2:18.2.2-1237.g646b8c7d.el8.x86_64 is already installed. 2024-04-25T20:08:04.776 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-04-25T20:08:04.777 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2024-04-25T20:08:04.777 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-04-25T20:08:04.839 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install python3-cephfs 2024-04-25T20:08:05.184 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:38 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:08:05.304 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-25T20:08:05.305 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:08:05.305 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repo Size 2024-04-25T20:08:05.305 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:08:05.305 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-04-25T20:08:05.305 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs-devel x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 31 k 2024-04-25T20:08:05.305 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:05.305 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-04-25T20:08:05.305 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:08:05.305 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2024-04-25T20:08:05.305 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:05.305 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 31 k 2024-04-25T20:08:05.306 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 139 k 2024-04-25T20:08:05.306 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-04-25T20:08:05.370 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:38 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:08:05.451 INFO:teuthology.orchestra.run.smithi039.stdout:Package python3-cephfs-2:18.2.2-1237.g646b8c7d.el8.x86_64 is already installed. 2024-04-25T20:08:05.494 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-04-25T20:08:05.495 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2024-04-25T20:08:05.495 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-04-25T20:08:05.559 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install python3-rbd 2024-04-25T20:08:05.807 INFO:teuthology.orchestra.run.smithi135.stdout:libcephfs-devel-18.2.2-1237.g646b8c7d.el8.x86_6 62 kB/s | 31 kB 00:00 2024-04-25T20:08:05.807 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:08:05.808 INFO:teuthology.orchestra.run.smithi135.stdout:Total 62 kB/s | 31 kB 00:00 2024-04-25T20:08:05.808 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-04-25T20:08:05.811 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-04-25T20:08:05.811 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-04-25T20:08:05.814 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-04-25T20:08:05.814 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-04-25T20:08:05.895 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-04-25T20:08:06.003 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephfs-devel-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:08:06.099 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:39 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:08:06.181 INFO:teuthology.orchestra.run.smithi039.stdout:Package python3-rbd-2:18.2.2-1237.g646b8c7d.el8.x86_64 is already installed. 2024-04-25T20:08:06.224 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-04-25T20:08:06.225 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2024-04-25T20:08:06.225 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-04-25T20:08:06.289 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install rbd-fuse 2024-04-25T20:08:06.338 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephfs-devel-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:08:06.544 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephfs-devel-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:08:06.545 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:06.545 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-04-25T20:08:06.545 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs-devel-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:08:06.545 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:06.545 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-25T20:08:06.711 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install librados2 2024-04-25T20:08:06.826 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:39 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:08:06.951 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-04-25T20:08:06.952 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:08:06.952 INFO:teuthology.orchestra.run.smithi039.stdout: Package Architecture Version Repository Size 2024-04-25T20:08:06.952 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:08:06.952 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-04-25T20:08:06.952 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-fuse x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 87 k 2024-04-25T20:08:06.952 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:06.952 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-04-25T20:08:06.952 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:08:06.952 INFO:teuthology.orchestra.run.smithi039.stdout:Install 1 Package 2024-04-25T20:08:06.952 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:06.952 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 87 k 2024-04-25T20:08:06.953 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 227 k 2024-04-25T20:08:06.953 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-04-25T20:08:07.220 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:40 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:08:07.298 INFO:teuthology.orchestra.run.smithi135.stdout:Package librados2-2:18.2.2-1237.g646b8c7d.el8.x86_64 is already installed. 2024-04-25T20:08:07.339 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-25T20:08:07.340 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2024-04-25T20:08:07.340 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-25T20:08:07.402 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install librbd1 2024-04-25T20:08:07.419 INFO:teuthology.orchestra.run.smithi039.stdout:rbd-fuse-18.2.2-1237.g646b8c7d.el8.x86_64.rpm 186 kB/s | 87 kB 00:00 2024-04-25T20:08:07.419 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:08:07.420 INFO:teuthology.orchestra.run.smithi039.stdout:Total 185 kB/s | 87 kB 00:00 2024-04-25T20:08:07.420 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-04-25T20:08:07.428 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-04-25T20:08:07.428 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-04-25T20:08:07.494 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-04-25T20:08:07.494 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-04-25T20:08:07.642 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-04-25T20:08:07.742 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-fuse-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:08:07.911 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:40 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:08:07.990 INFO:teuthology.orchestra.run.smithi135.stdout:Package librbd1-2:18.2.2-1237.g646b8c7d.el8.x86_64 is already installed. 2024-04-25T20:08:08.030 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-25T20:08:08.031 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2024-04-25T20:08:08.031 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-25T20:08:08.080 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: rbd-fuse-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:08:08.093 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install python3-rados 2024-04-25T20:08:08.336 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-fuse-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:08:08.337 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:08.337 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-04-25T20:08:08.337 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-fuse-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:08:08.337 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:08.337 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-04-25T20:08:08.564 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install rbd-mirror 2024-04-25T20:08:08.602 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:41 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:08:08.680 INFO:teuthology.orchestra.run.smithi135.stdout:Package python3-rados-2:18.2.2-1237.g646b8c7d.el8.x86_64 is already installed. 2024-04-25T20:08:08.720 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-25T20:08:08.721 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2024-04-25T20:08:08.721 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-25T20:08:08.786 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install python3-rgw 2024-04-25T20:08:09.107 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:42 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:08:09.234 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-04-25T20:08:09.235 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:08:09.235 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repo Size 2024-04-25T20:08:09.235 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:08:09.235 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-04-25T20:08:09.235 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-mirror x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 3.0 M 2024-04-25T20:08:09.235 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:09.235 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-04-25T20:08:09.235 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:08:09.235 INFO:teuthology.orchestra.run.smithi039.stdout:Install 1 Package 2024-04-25T20:08:09.235 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:09.235 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 3.0 M 2024-04-25T20:08:09.235 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 12 M 2024-04-25T20:08:09.235 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-04-25T20:08:09.302 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:42 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:08:09.381 INFO:teuthology.orchestra.run.smithi135.stdout:Package python3-rgw-2:18.2.2-1237.g646b8c7d.el8.x86_64 is already installed. 2024-04-25T20:08:09.421 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-25T20:08:09.422 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2024-04-25T20:08:09.423 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-25T20:08:09.485 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install python3-cephfs 2024-04-25T20:08:09.995 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:42 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:08:10.073 INFO:teuthology.orchestra.run.smithi135.stdout:Package python3-cephfs-2:18.2.2-1237.g646b8c7d.el8.x86_64 is already installed. 2024-04-25T20:08:10.113 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-25T20:08:10.114 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2024-04-25T20:08:10.114 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-25T20:08:10.176 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install python3-rbd 2024-04-25T20:08:10.693 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:43 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:08:10.776 INFO:teuthology.orchestra.run.smithi135.stdout:Package python3-rbd-2:18.2.2-1237.g646b8c7d.el8.x86_64 is already installed. 2024-04-25T20:08:10.817 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-25T20:08:10.818 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2024-04-25T20:08:10.818 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-25T20:08:10.884 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install rbd-fuse 2024-04-25T20:08:11.395 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:44 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:08:11.514 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-25T20:08:11.515 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:08:11.515 INFO:teuthology.orchestra.run.smithi135.stdout: Package Architecture Version Repository Size 2024-04-25T20:08:11.515 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:08:11.515 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-04-25T20:08:11.515 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-fuse x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 87 k 2024-04-25T20:08:11.515 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:11.515 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-04-25T20:08:11.515 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:08:11.515 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2024-04-25T20:08:11.515 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:11.515 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 87 k 2024-04-25T20:08:11.515 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 227 k 2024-04-25T20:08:11.516 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-04-25T20:08:11.607 INFO:teuthology.orchestra.run.smithi039.stdout:rbd-mirror-18.2.2-1237.g646b8c7d.el8.x86_64.rpm 1.3 MB/s | 3.0 MB 00:02 2024-04-25T20:08:11.607 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:08:11.608 INFO:teuthology.orchestra.run.smithi039.stdout:Total 1.3 MB/s | 3.0 MB 00:02 2024-04-25T20:08:11.608 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-04-25T20:08:11.620 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-04-25T20:08:11.620 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-04-25T20:08:11.714 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-04-25T20:08:11.714 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-04-25T20:08:12.010 INFO:teuthology.orchestra.run.smithi135.stdout:rbd-fuse-18.2.2-1237.g646b8c7d.el8.x86_64.rpm 175 kB/s | 87 kB 00:00 2024-04-25T20:08:12.011 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:08:12.011 INFO:teuthology.orchestra.run.smithi135.stdout:Total 175 kB/s | 87 kB 00:00 2024-04-25T20:08:12.011 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-04-25T20:08:12.018 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-04-25T20:08:12.018 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-04-25T20:08:12.079 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-04-25T20:08:12.079 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-04-25T20:08:12.153 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-04-25T20:08:12.220 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-04-25T20:08:12.232 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-mirror-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:08:12.258 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:08:12.258 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:08:12.258 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T20:08:12.258 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T20:08:12.258 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T20:08:12.258 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:12.322 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-fuse-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:08:12.676 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: rbd-fuse-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:08:12.900 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-fuse-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:08:12.900 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:12.900 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-04-25T20:08:12.900 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-fuse-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:08:12.900 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:12.900 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-25T20:08:13.135 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install rbd-mirror 2024-04-25T20:08:13.328 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-mirror-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:08:13.328 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:13.328 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-04-25T20:08:13.328 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-mirror-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:08:13.328 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:13.328 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-04-25T20:08:13.557 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install rbd-nbd 2024-04-25T20:08:13.658 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:46 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:08:13.779 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-25T20:08:13.780 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:08:13.780 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repo Size 2024-04-25T20:08:13.780 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:08:13.780 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-04-25T20:08:13.780 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-mirror x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 3.0 M 2024-04-25T20:08:13.780 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:13.780 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-04-25T20:08:13.780 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:08:13.781 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2024-04-25T20:08:13.781 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:13.781 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 3.0 M 2024-04-25T20:08:13.781 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 12 M 2024-04-25T20:08:13.781 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-04-25T20:08:14.101 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:47 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:08:14.225 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-04-25T20:08:14.226 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:08:14.226 INFO:teuthology.orchestra.run.smithi039.stdout: Package Architecture Version Repository Size 2024-04-25T20:08:14.226 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:08:14.226 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-04-25T20:08:14.226 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-nbd x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 172 k 2024-04-25T20:08:14.226 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:14.227 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-04-25T20:08:14.227 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-04-25T20:08:14.227 INFO:teuthology.orchestra.run.smithi039.stdout:Install 1 Package 2024-04-25T20:08:14.227 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:14.227 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 172 k 2024-04-25T20:08:14.227 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 494 k 2024-04-25T20:08:14.227 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-04-25T20:08:14.761 INFO:teuthology.orchestra.run.smithi039.stdout:rbd-nbd-18.2.2-1237.g646b8c7d.el8.x86_64.rpm 322 kB/s | 172 kB 00:00 2024-04-25T20:08:14.762 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:08:14.762 INFO:teuthology.orchestra.run.smithi039.stdout:Total 321 kB/s | 172 kB 00:00 2024-04-25T20:08:14.762 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-04-25T20:08:14.771 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-04-25T20:08:14.771 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-04-25T20:08:14.840 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-04-25T20:08:14.840 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-04-25T20:08:14.995 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-04-25T20:08:15.119 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-nbd-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:08:15.188 INFO:teuthology.orchestra.run.smithi135.stdout:rbd-mirror-18.2.2-1237.g646b8c7d.el8.x86_64.rpm 2.1 MB/s | 3.0 MB 00:01 2024-04-25T20:08:15.189 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:08:15.189 INFO:teuthology.orchestra.run.smithi135.stdout:Total 2.1 MB/s | 3.0 MB 00:01 2024-04-25T20:08:15.189 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-04-25T20:08:15.199 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-04-25T20:08:15.199 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-04-25T20:08:15.284 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-04-25T20:08:15.284 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-04-25T20:08:15.506 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: rbd-nbd-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:08:15.713 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-04-25T20:08:15.772 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-mirror-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:08:15.776 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-nbd-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:08:15.776 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:15.776 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-04-25T20:08:15.776 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-nbd-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:08:15.776 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:15.776 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-04-25T20:08:15.796 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:08:15.796 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:08:15.796 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T20:08:15.796 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T20:08:15.796 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T20:08:15.797 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:15.993 DEBUG:teuthology.parallel:result is None 2024-04-25T20:08:16.782 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-mirror-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:08:16.783 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:16.783 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-04-25T20:08:16.783 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-mirror-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:08:16.783 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:16.783 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-25T20:08:17.009 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install rbd-nbd 2024-04-25T20:08:17.547 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:50 ago on Thu 25 Apr 2024 08:06:27 PM UTC. 2024-04-25T20:08:17.671 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-25T20:08:17.672 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:08:17.672 INFO:teuthology.orchestra.run.smithi135.stdout: Package Architecture Version Repository Size 2024-04-25T20:08:17.672 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:08:17.673 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-04-25T20:08:17.673 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-nbd x86_64 2:18.2.2-1237.g646b8c7d.el8 ceph 172 k 2024-04-25T20:08:17.673 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:17.673 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-04-25T20:08:17.673 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-25T20:08:17.673 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2024-04-25T20:08:17.673 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:17.673 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 172 k 2024-04-25T20:08:17.673 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 494 k 2024-04-25T20:08:17.673 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-04-25T20:08:17.900 INFO:teuthology.orchestra.run.smithi135.stdout:rbd-nbd-18.2.2-1237.g646b8c7d.el8.x86_64.rpm 761 kB/s | 172 kB 00:00 2024-04-25T20:08:17.900 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:08:17.901 INFO:teuthology.orchestra.run.smithi135.stdout:Total 755 kB/s | 172 kB 00:00 2024-04-25T20:08:17.901 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-04-25T20:08:17.908 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-04-25T20:08:17.909 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-04-25T20:08:17.971 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-04-25T20:08:17.971 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-04-25T20:08:18.122 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-04-25T20:08:18.237 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-nbd-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:08:18.623 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: rbd-nbd-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:08:18.882 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-nbd-2:18.2.2-1237.g646b8c7d.el8.x86_64 1/1 2024-04-25T20:08:18.882 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:18.882 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-04-25T20:08:18.882 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-nbd-2:18.2.2-1237.g646b8c7d.el8.x86_64 2024-04-25T20:08:18.882 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:18.882 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-25T20:08:19.110 DEBUG:teuthology.parallel:result is None 2024-04-25T20:08:19.111 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-25T20:08:19.252 DEBUG:teuthology.orchestra.run.smithi039:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T20:08:19.278 INFO:teuthology.orchestra.run.smithi039.stdout:18.2.2-1237.g646b8c7d.el8 2024-04-25T20:08:19.279 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1237.g646b8c7d.el8 2024-04-25T20:08:19.279 INFO:teuthology.task.install:The correct ceph version 18.2.2-1237.g646b8c7d is installed. 2024-04-25T20:08:19.281 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-25T20:08:19.421 DEBUG:teuthology.orchestra.run.smithi135:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T20:08:19.442 INFO:teuthology.orchestra.run.smithi135.stdout:18.2.2-1237.g646b8c7d.el8 2024-04-25T20:08:19.442 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1237.g646b8c7d.el8 2024-04-25T20:08:19.442 INFO:teuthology.task.install:The correct ceph version 18.2.2-1237.g646b8c7d is installed. 2024-04-25T20:08:19.444 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T20:08:19.444 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-04-25T20:08:19.444 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T20:08:19.474 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-25T20:08:19.474 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T20:08:19.506 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T20:08:19.506 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-04-25T20:08:19.506 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T20:08:19.543 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T20:08:19.610 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-25T20:08:19.610 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T20:08:19.634 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T20:08:19.697 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T20:08:19.697 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-04-25T20:08:19.697 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T20:08:19.723 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T20:08:19.790 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-25T20:08:19.790 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T20:08:19.813 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T20:08:19.876 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T20:08:19.876 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-04-25T20:08:19.876 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T20:08:19.902 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T20:08:19.968 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-25T20:08:19.968 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T20:08:19.992 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T20:08:20.056 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T20:08:20.143 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T20:08:20.143 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 777 /var/log/ceph 2024-04-25T20:08:20.147 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 777 /var/log/ceph 2024-04-25T20:08:20.170 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T20:08:20.170 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T20:08:20.213 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T20:08:20.236 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T20:08:20.236 DEBUG:teuthology.orchestra.run.smithi039:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T20:08:20.279 DEBUG:teuthology.orchestra.run.smithi135:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T20:08:20.311 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T20:08:20.312 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T20:08:20.312 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-23_22:16:02-rbd-reef-distro-default-smithi/7670822', 'branch': 'reef', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/journal}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7670822', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-04-23_22:16:02-rbd-reef-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '646b8c7d293d8ec31a40c69f7b3237b19c1be24d'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '646b8c7d293d8ec31a40c69f7b3237b19c1be24d'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': '19b26365dac54ee709df8c5dc2a7b7e89a50aba5'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 930, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 2987, 'sha1': '646b8c7d293d8ec31a40c69f7b3237b19c1be24d', 'sleep_before_teardown': 0, 'subset': '17/128', 'suite': 'rbd', 'suite_branch': 'reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_19b26365dac54ee709df8c5dc2a7b7e89a50aba5/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '19b26365dac54ee709df8c5dc2a7b7e89a50aba5', 'targets': {'smithi039.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPx7DNlgqpi1ZwRCwkvSCxi2BlSMk8UY6pu4oQo7xnMWVcb1sBPhUbx41iRbpN+5U6ckuXt/05601eWN/K2B+Jw=', 'smithi135.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBONrHp0x4i8dM+RTeVczM1nKavKGfa39I1L1k+GaFeYF9LPF9lEqkUkFW7f1dKw8nW8rHjX+IdiF6GXTDpdianQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'workunit': {'clients': {'client.0': ['rbd/journal.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-23_22:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T20:08:20.312 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T20:08:20.344 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T20:08:20.359 DEBUG:teuthology.orchestra.run.smithi039:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T20:08:20.400 DEBUG:teuthology.orchestra.run.smithi135:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T20:08:20.423 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-04-25T20:08:20.423 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T20:08:20.478 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T20:08:20.479 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_1 2024-04-25T20:08:20.534 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T20:08:20.534 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:08:20.534 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-25T20:08:20.534 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:08:20.534 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:08:20.534 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-04-25 20:08:12.530523432 +0000 2024-04-25T20:08:20.534 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-04-25 20:05:50.749058511 +0000 2024-04-25T20:08:20.535 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-04-25 20:05:50.749058511 +0000 2024-04-25T20:08:20.535 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-04-25 20:05:50.749058511 +0000 2024-04-25T20:08:20.535 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T20:08:20.597 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-04-25T20:08:20.597 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-04-25T20:08:20.597 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000282354 s, 1.8 MB/s 2024-04-25T20:08:20.598 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T20:08:20.654 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_2 2024-04-25T20:08:20.709 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T20:08:20.709 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:08:20.709 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-25T20:08:20.709 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:08:20.709 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:08:20.710 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-04-25 20:08:12.530523432 +0000 2024-04-25T20:08:20.710 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-04-25 20:05:51.115050458 +0000 2024-04-25T20:08:20.710 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-04-25 20:05:51.115050458 +0000 2024-04-25T20:08:20.710 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-04-25 20:05:51.115050458 +0000 2024-04-25T20:08:20.710 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T20:08:20.772 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-04-25T20:08:20.772 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-04-25T20:08:20.772 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000236983 s, 2.2 MB/s 2024-04-25T20:08:20.773 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T20:08:20.829 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_3 2024-04-25T20:08:20.884 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T20:08:20.884 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:08:20.884 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T20:08:20.884 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:08:20.885 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:08:20.885 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-04-25 20:08:12.531523411 +0000 2024-04-25T20:08:20.885 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-04-25 20:05:51.463042800 +0000 2024-04-25T20:08:20.885 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-04-25 20:05:51.463042800 +0000 2024-04-25T20:08:20.885 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-04-25 20:05:51.463042800 +0000 2024-04-25T20:08:20.885 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T20:08:20.948 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-04-25T20:08:20.948 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-04-25T20:08:20.948 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000415318 s, 1.2 MB/s 2024-04-25T20:08:20.949 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T20:08:21.006 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_4 2024-04-25T20:08:21.061 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T20:08:21.062 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:08:21.062 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T20:08:21.062 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:08:21.062 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:08:21.062 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-04-25 20:08:12.531523411 +0000 2024-04-25T20:08:21.062 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-04-25 20:05:51.812035120 +0000 2024-04-25T20:08:21.062 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-04-25 20:05:51.812035120 +0000 2024-04-25T20:08:21.062 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-04-25 20:05:51.812035120 +0000 2024-04-25T20:08:21.062 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T20:08:21.125 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-04-25T20:08:21.125 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-04-25T20:08:21.125 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000270822 s, 1.9 MB/s 2024-04-25T20:08:21.126 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T20:08:21.182 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T20:08:21.182 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-25T20:08:21.183 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T20:08:21.200 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T20:08:21.200 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_1 2024-04-25T20:08:21.257 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T20:08:21.257 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:08:21.257 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-25T20:08:21.257 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:08:21.257 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:08:21.258 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-04-25 20:08:16.042864575 +0000 2024-04-25T20:08:21.258 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-04-25 20:05:46.881400347 +0000 2024-04-25T20:08:21.258 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-04-25 20:05:46.881400347 +0000 2024-04-25T20:08:21.258 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-04-25 20:05:46.881400347 +0000 2024-04-25T20:08:21.258 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T20:08:21.324 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-04-25T20:08:21.325 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-04-25T20:08:21.325 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000211706 s, 2.4 MB/s 2024-04-25T20:08:21.326 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T20:08:21.385 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_2 2024-04-25T20:08:21.443 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T20:08:21.444 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:08:21.444 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T20:08:21.444 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:08:21.444 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:08:21.444 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-04-25 20:08:16.043864552 +0000 2024-04-25T20:08:21.444 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-04-25 20:05:47.216392664 +0000 2024-04-25T20:08:21.444 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-04-25 20:05:47.216392664 +0000 2024-04-25T20:08:21.444 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-04-25 20:05:47.216392664 +0000 2024-04-25T20:08:21.444 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T20:08:21.509 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-04-25T20:08:21.509 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-04-25T20:08:21.509 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000245948 s, 2.1 MB/s 2024-04-25T20:08:21.510 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T20:08:21.568 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_3 2024-04-25T20:08:21.626 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T20:08:21.626 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:08:21.626 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T20:08:21.626 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:08:21.626 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:08:21.626 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-04-25 20:08:16.043864552 +0000 2024-04-25T20:08:21.626 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-04-25 20:05:47.556384865 +0000 2024-04-25T20:08:21.626 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-04-25 20:05:47.556384865 +0000 2024-04-25T20:08:21.627 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-04-25 20:05:47.556384865 +0000 2024-04-25T20:08:21.627 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T20:08:21.690 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-04-25T20:08:21.690 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-04-25T20:08:21.690 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000223129 s, 2.3 MB/s 2024-04-25T20:08:21.691 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T20:08:21.748 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_4 2024-04-25T20:08:21.805 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T20:08:21.806 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:08:21.806 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-25T20:08:21.806 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:08:21.806 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:08:21.806 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-04-25 20:08:16.043864552 +0000 2024-04-25T20:08:21.806 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-04-25 20:05:47.912376700 +0000 2024-04-25T20:08:21.806 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-04-25 20:05:47.912376700 +0000 2024-04-25T20:08:21.806 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-04-25 20:05:47.912376700 +0000 2024-04-25T20:08:21.806 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T20:08:21.869 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-04-25T20:08:21.869 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-04-25T20:08:21.869 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000189342 s, 2.7 MB/s 2024-04-25T20:08:21.870 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T20:08:21.928 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T20:08:21.928 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi039.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi135.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T20:08:21.928 INFO:tasks.ceph:Generating config... 2024-04-25T20:08:21.929 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-04-25T20:08:21.929 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-04-25T20:08:21.930 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-04-25T20:08:21.930 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T20:08:21.930 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T20:08:21.930 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T20:08:21.930 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T20:08:21.930 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T20:08:21.930 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T20:08:21.930 INFO:tasks.ceph:[osd] bluestore compression algorithm = lz4 2024-04-25T20:08:21.930 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-04-25T20:08:21.930 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T20:08:21.930 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-25T20:08:21.930 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-25T20:08:21.930 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T20:08:21.930 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T20:08:21.931 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-25T20:08:21.931 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T20:08:21.931 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T20:08:21.931 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T20:08:21.931 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-04-25T20:08:21.931 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T20:08:21.931 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-04-25T20:08:21.931 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T20:08:21.931 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-04-25T20:08:21.931 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T20:08:21.931 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T20:08:21.991 INFO:teuthology.orchestra.run.smithi039.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T20:08:21.994 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T20:08:22.037 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T20:08:22.062 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.39'), ('mon.c', '[v2:172.21.15.39:3301,v1:172.21.15.39:6790]'), ('mon.b', '172.21.15.135')] 2024-04-25T20:08:22.062 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.39,[v2:172.21.15.39:3301,v1:172.21.15.39:6790],172.21.15.135', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-04-25T20:08:22.063 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-04-25T20:08:22.063 DEBUG:teuthology.orchestra.run.smithi039:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T20:08:22.118 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.39 --addv c '[v2:172.21.15.39:3301,v1:172.21.15.39:6790]' --add b 172.21.15.135 --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T20:08:22.204 INFO:teuthology.orchestra.run.smithi039.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T20:08:22.204 INFO:teuthology.orchestra.run.smithi039.stdout:monmaptool: generated fsid 975ceafd-555d-4f71-81e4-1f70364da00f 2024-04-25T20:08:22.204 INFO:teuthology.orchestra.run.smithi039.stdout:setting min_mon_release = pacific 2024-04-25T20:08:22.204 INFO:teuthology.orchestra.run.smithi039.stdout:epoch 0 2024-04-25T20:08:22.204 INFO:teuthology.orchestra.run.smithi039.stdout:fsid 975ceafd-555d-4f71-81e4-1f70364da00f 2024-04-25T20:08:22.204 INFO:teuthology.orchestra.run.smithi039.stdout:last_changed 2024-04-25T20:08:22.204621+0000 2024-04-25T20:08:22.204 INFO:teuthology.orchestra.run.smithi039.stdout:created 2024-04-25T20:08:22.204621+0000 2024-04-25T20:08:22.204 INFO:teuthology.orchestra.run.smithi039.stdout:min_mon_release 16 (pacific) 2024-04-25T20:08:22.205 INFO:teuthology.orchestra.run.smithi039.stdout:election_strategy: 1 2024-04-25T20:08:22.205 INFO:teuthology.orchestra.run.smithi039.stdout:0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-04-25T20:08:22.205 INFO:teuthology.orchestra.run.smithi039.stdout:1: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.b 2024-04-25T20:08:22.205 INFO:teuthology.orchestra.run.smithi039.stdout:2: [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] mon.c 2024-04-25T20:08:22.205 INFO:teuthology.orchestra.run.smithi039.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T20:08:22.206 DEBUG:teuthology.orchestra.run.smithi039:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T20:08:22.221 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 975ceafd-555d-4f71-81e4-1f70364da00f... 2024-04-25T20:08:22.222 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T20:08:22.264 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T20:08:22.302 INFO:teuthology.orchestra.run.smithi039.stdout:[global] 2024-04-25T20:08:22.302 INFO:teuthology.orchestra.run.smithi039.stdout: chdir = "" 2024-04-25T20:08:22.303 INFO:teuthology.orchestra.run.smithi039.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T20:08:22.303 INFO:teuthology.orchestra.run.smithi039.stdout: auth supported = cephx 2024-04-25T20:08:22.303 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.303 INFO:teuthology.orchestra.run.smithi039.stdout: filestore xattr use omap = true 2024-04-25T20:08:22.303 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.303 INFO:teuthology.orchestra.run.smithi039.stdout: mon clock drift allowed = 1.000 2024-04-25T20:08:22.303 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.303 INFO:teuthology.orchestra.run.smithi039.stdout: osd crush chooseleaf type = 0 2024-04-25T20:08:22.303 INFO:teuthology.orchestra.run.smithi039.stdout: auth debug = true 2024-04-25T20:08:22.303 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.303 INFO:teuthology.orchestra.run.smithi039.stdout: ms die on old message = true 2024-04-25T20:08:22.303 INFO:teuthology.orchestra.run.smithi039.stdout: ms die on bug = true 2024-04-25T20:08:22.303 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.303 INFO:teuthology.orchestra.run.smithi039.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T20:08:22.303 INFO:teuthology.orchestra.run.smithi039.stdout: mon pg warn max object skew = 0 2024-04-25T20:08:22.304 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.304 INFO:teuthology.orchestra.run.smithi039.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T20:08:22.304 INFO:teuthology.orchestra.run.smithi039.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T20:08:22.304 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.304 INFO:teuthology.orchestra.run.smithi039.stdout: osd pool default size = 2 2024-04-25T20:08:22.304 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.304 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd allow primary affinity = true 2024-04-25T20:08:22.304 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd allow pg remap = true 2024-04-25T20:08:22.304 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on legacy crush tunables = false 2024-04-25T20:08:22.304 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on crush straw calc version zero = false 2024-04-25T20:08:22.304 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on no sortbitwise = false 2024-04-25T20:08:22.304 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on osd down out interval zero = false 2024-04-25T20:08:22.304 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on too few osds = false 2024-04-25T20:08:22.304 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T20:08:22.304 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T20:08:22.305 INFO:teuthology.orchestra.run.smithi039.stdout: mon_allow_pool_size_one = true 2024-04-25T20:08:22.305 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.305 INFO:teuthology.orchestra.run.smithi039.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T20:08:22.305 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.305 INFO:teuthology.orchestra.run.smithi039.stdout: osd default data pool replay window = 5 2024-04-25T20:08:22.305 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.305 INFO:teuthology.orchestra.run.smithi039.stdout: mon allow pool delete = true 2024-04-25T20:08:22.305 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.305 INFO:teuthology.orchestra.run.smithi039.stdout: mon cluster log file level = debug 2024-04-25T20:08:22.305 INFO:teuthology.orchestra.run.smithi039.stdout: debug asserts on shutdown = true 2024-04-25T20:08:22.305 INFO:teuthology.orchestra.run.smithi039.stdout: mon health detail to clog = false 2024-04-25T20:08:22.305 INFO:teuthology.orchestra.run.smithi039.stdout: mon host = "172.21.15.39,[v2:172.21.15.39:3301,v1:172.21.15.39:6790],172.21.15.135" 2024-04-25T20:08:22.305 INFO:teuthology.orchestra.run.smithi039.stdout: mon client directed command retry = 5 2024-04-25T20:08:22.305 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on pool no app = False 2024-04-25T20:08:22.305 INFO:teuthology.orchestra.run.smithi039.stdout: ms inject socket failures = 5000 2024-04-25T20:08:22.306 INFO:teuthology.orchestra.run.smithi039.stdout: fsid = 975ceafd-555d-4f71-81e4-1f70364da00f 2024-04-25T20:08:22.306 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.306 INFO:teuthology.orchestra.run.smithi039.stdout:[osd] 2024-04-25T20:08:22.306 INFO:teuthology.orchestra.run.smithi039.stdout: osd journal size = 100 2024-04-25T20:08:22.306 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.306 INFO:teuthology.orchestra.run.smithi039.stdout: osd scrub load threshold = 5.0 2024-04-25T20:08:22.306 INFO:teuthology.orchestra.run.smithi039.stdout: osd scrub max interval = 600 2024-04-25T20:08:22.306 INFO:teuthology.orchestra.run.smithi039.stdout: osd mclock profile = high_recovery_ops 2024-04-25T20:08:22.306 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.306 INFO:teuthology.orchestra.run.smithi039.stdout: osd recover clone overlap = true 2024-04-25T20:08:22.306 INFO:teuthology.orchestra.run.smithi039.stdout: osd recovery max chunk = 1048576 2024-04-25T20:08:22.307 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.307 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug shutdown = true 2024-04-25T20:08:22.307 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug op order = true 2024-04-25T20:08:22.307 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug verify stray on activate = true 2024-04-25T20:08:22.307 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.307 INFO:teuthology.orchestra.run.smithi039.stdout: osd open classes on start = true 2024-04-25T20:08:22.307 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug pg log writeout = true 2024-04-25T20:08:22.307 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.307 INFO:teuthology.orchestra.run.smithi039.stdout: osd deep scrub update digest min age = 30 2024-04-25T20:08:22.307 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.307 INFO:teuthology.orchestra.run.smithi039.stdout: osd map max advance = 10 2024-04-25T20:08:22.307 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.307 INFO:teuthology.orchestra.run.smithi039.stdout: journal zero on create = true 2024-04-25T20:08:22.307 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.307 INFO:teuthology.orchestra.run.smithi039.stdout: filestore ondisk finisher threads = 3 2024-04-25T20:08:22.308 INFO:teuthology.orchestra.run.smithi039.stdout: filestore apply finisher threads = 3 2024-04-25T20:08:22.308 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.308 INFO:teuthology.orchestra.run.smithi039.stdout: bdev debug aio = true 2024-04-25T20:08:22.308 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug misdirected ops = true 2024-04-25T20:08:22.308 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore block size = 96636764160 2024-04-25T20:08:22.308 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore compression algorithm = lz4 2024-04-25T20:08:22.308 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore compression mode = aggressive 2024-04-25T20:08:22.308 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore fsck on mount = True 2024-04-25T20:08:22.308 INFO:teuthology.orchestra.run.smithi039.stdout: debug bluefs = 1/20 2024-04-25T20:08:22.308 INFO:teuthology.orchestra.run.smithi039.stdout: debug bluestore = 1/20 2024-04-25T20:08:22.308 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-04-25T20:08:22.308 INFO:teuthology.orchestra.run.smithi039.stdout: debug osd = 20 2024-04-25T20:08:22.308 INFO:teuthology.orchestra.run.smithi039.stdout: debug rocksdb = 4/10 2024-04-25T20:08:22.309 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T20:08:22.309 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd full ratio = 0.9 2024-04-25T20:08:22.309 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd nearfull ratio = 0.8 2024-04-25T20:08:22.309 INFO:teuthology.orchestra.run.smithi039.stdout: osd blocked scrub grace period = 3600 2024-04-25T20:08:22.309 INFO:teuthology.orchestra.run.smithi039.stdout: osd failsafe full ratio = 0.95 2024-04-25T20:08:22.309 INFO:teuthology.orchestra.run.smithi039.stdout: osd max markdown count = 1000 2024-04-25T20:08:22.309 INFO:teuthology.orchestra.run.smithi039.stdout: osd objectstore = bluestore 2024-04-25T20:08:22.309 INFO:teuthology.orchestra.run.smithi039.stdout: osd shutdown pgref assert = True 2024-04-25T20:08:22.309 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.309 INFO:teuthology.orchestra.run.smithi039.stdout:[mgr] 2024-04-25T20:08:22.309 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-04-25T20:08:22.309 INFO:teuthology.orchestra.run.smithi039.stdout: debug mgr = 20 2024-04-25T20:08:22.309 INFO:teuthology.orchestra.run.smithi039.stdout: debug mon = 20 2024-04-25T20:08:22.309 INFO:teuthology.orchestra.run.smithi039.stdout: debug auth = 20 2024-04-25T20:08:22.309 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min pgs per osd = 4 2024-04-25T20:08:22.310 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min bytes per osd = 10 2024-04-25T20:08:22.310 INFO:teuthology.orchestra.run.smithi039.stdout: mgr/telemetry/nag = false 2024-04-25T20:08:22.310 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.310 INFO:teuthology.orchestra.run.smithi039.stdout:[mon] 2024-04-25T20:08:22.310 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-04-25T20:08:22.310 INFO:teuthology.orchestra.run.smithi039.stdout: debug mon = 20 2024-04-25T20:08:22.310 INFO:teuthology.orchestra.run.smithi039.stdout: debug paxos = 20 2024-04-25T20:08:22.310 INFO:teuthology.orchestra.run.smithi039.stdout: debug auth = 20 2024-04-25T20:08:22.310 INFO:teuthology.orchestra.run.smithi039.stdout: mon data avail warn = 5 2024-04-25T20:08:22.310 INFO:teuthology.orchestra.run.smithi039.stdout: mon mgr mkfs grace = 240 2024-04-25T20:08:22.310 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min pgs per osd = 4 2024-04-25T20:08:22.310 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd reporter subtree level = osd 2024-04-25T20:08:22.310 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd prime pg temp = true 2024-04-25T20:08:22.310 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min bytes per osd = 10 2024-04-25T20:08:22.311 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.311 INFO:teuthology.orchestra.run.smithi039.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T20:08:22.311 INFO:teuthology.orchestra.run.smithi039.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T20:08:22.311 INFO:teuthology.orchestra.run.smithi039.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T20:08:22.311 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.311 INFO:teuthology.orchestra.run.smithi039.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T20:08:22.311 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T20:08:22.311 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T20:08:22.311 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.311 INFO:teuthology.orchestra.run.smithi039.stdout: # 1m isn't quite enough 2024-04-25T20:08:22.311 INFO:teuthology.orchestra.run.smithi039.stdout: mon_down_mkfs_grace = 2m 2024-04-25T20:08:22.311 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.311 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_filestore_osds = false 2024-04-25T20:08:22.311 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:08:22.311 INFO:teuthology.orchestra.run.smithi039.stdout:[client] 2024-04-25T20:08:22.311 INFO:teuthology.orchestra.run.smithi039.stdout: rgw cache enabled = true 2024-04-25T20:08:22.312 INFO:teuthology.orchestra.run.smithi039.stdout: rgw enable ops log = true 2024-04-25T20:08:22.312 INFO:teuthology.orchestra.run.smithi039.stdout: rgw enable usage log = true 2024-04-25T20:08:22.312 INFO:teuthology.orchestra.run.smithi039.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T20:08:22.312 INFO:teuthology.orchestra.run.smithi039.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T20:08:22.312 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.a] 2024-04-25T20:08:22.312 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.c] 2024-04-25T20:08:22.312 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.b] 2024-04-25T20:08:22.312 INFO:teuthology.orchestra.run.smithi135.stdout:[global] 2024-04-25T20:08:22.312 INFO:teuthology.orchestra.run.smithi135.stdout: chdir = "" 2024-04-25T20:08:22.313 INFO:teuthology.orchestra.run.smithi135.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T20:08:22.313 INFO:teuthology.orchestra.run.smithi135.stdout: auth supported = cephx 2024-04-25T20:08:22.313 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.313 INFO:teuthology.orchestra.run.smithi135.stdout: filestore xattr use omap = true 2024-04-25T20:08:22.313 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.313 INFO:teuthology.orchestra.run.smithi135.stdout: mon clock drift allowed = 1.000 2024-04-25T20:08:22.313 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.313 INFO:teuthology.orchestra.run.smithi135.stdout: osd crush chooseleaf type = 0 2024-04-25T20:08:22.313 INFO:teuthology.orchestra.run.smithi135.stdout: auth debug = true 2024-04-25T20:08:22.313 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.313 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on old message = true 2024-04-25T20:08:22.313 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on bug = true 2024-04-25T20:08:22.314 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.314 INFO:teuthology.orchestra.run.smithi135.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T20:08:22.314 INFO:teuthology.orchestra.run.smithi135.stdout: mon pg warn max object skew = 0 2024-04-25T20:08:22.314 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.314 INFO:teuthology.orchestra.run.smithi135.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T20:08:22.314 INFO:teuthology.orchestra.run.smithi135.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T20:08:22.314 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.314 INFO:teuthology.orchestra.run.smithi135.stdout: osd pool default size = 2 2024-04-25T20:08:22.314 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.314 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd allow primary affinity = true 2024-04-25T20:08:22.314 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd allow pg remap = true 2024-04-25T20:08:22.314 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on legacy crush tunables = false 2024-04-25T20:08:22.315 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on crush straw calc version zero = false 2024-04-25T20:08:22.315 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on no sortbitwise = false 2024-04-25T20:08:22.315 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on osd down out interval zero = false 2024-04-25T20:08:22.315 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on too few osds = false 2024-04-25T20:08:22.315 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T20:08:22.315 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T20:08:22.315 INFO:teuthology.orchestra.run.smithi135.stdout: mon_allow_pool_size_one = true 2024-04-25T20:08:22.315 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.315 INFO:teuthology.orchestra.run.smithi135.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T20:08:22.315 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.315 INFO:teuthology.orchestra.run.smithi135.stdout: osd default data pool replay window = 5 2024-04-25T20:08:22.315 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.315 INFO:teuthology.orchestra.run.smithi135.stdout: mon allow pool delete = true 2024-04-25T20:08:22.315 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.315 INFO:teuthology.orchestra.run.smithi135.stdout: mon cluster log file level = debug 2024-04-25T20:08:22.316 INFO:teuthology.orchestra.run.smithi135.stdout: debug asserts on shutdown = true 2024-04-25T20:08:22.316 INFO:teuthology.orchestra.run.smithi135.stdout: mon health detail to clog = false 2024-04-25T20:08:22.316 INFO:teuthology.orchestra.run.smithi135.stdout: mon host = "172.21.15.39,[v2:172.21.15.39:3301,v1:172.21.15.39:6790],172.21.15.135" 2024-04-25T20:08:22.316 INFO:teuthology.orchestra.run.smithi135.stdout: mon client directed command retry = 5 2024-04-25T20:08:22.316 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on pool no app = False 2024-04-25T20:08:22.316 INFO:teuthology.orchestra.run.smithi135.stdout: ms inject socket failures = 5000 2024-04-25T20:08:22.316 INFO:teuthology.orchestra.run.smithi135.stdout: fsid = 975ceafd-555d-4f71-81e4-1f70364da00f 2024-04-25T20:08:22.316 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.316 INFO:teuthology.orchestra.run.smithi135.stdout:[osd] 2024-04-25T20:08:22.316 INFO:teuthology.orchestra.run.smithi135.stdout: osd journal size = 100 2024-04-25T20:08:22.316 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.316 INFO:teuthology.orchestra.run.smithi135.stdout: osd scrub load threshold = 5.0 2024-04-25T20:08:22.316 INFO:teuthology.orchestra.run.smithi135.stdout: osd scrub max interval = 600 2024-04-25T20:08:22.316 INFO:teuthology.orchestra.run.smithi135.stdout: osd mclock profile = high_recovery_ops 2024-04-25T20:08:22.316 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.317 INFO:teuthology.orchestra.run.smithi135.stdout: osd recover clone overlap = true 2024-04-25T20:08:22.317 INFO:teuthology.orchestra.run.smithi135.stdout: osd recovery max chunk = 1048576 2024-04-25T20:08:22.317 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.317 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug shutdown = true 2024-04-25T20:08:22.317 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug op order = true 2024-04-25T20:08:22.317 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug verify stray on activate = true 2024-04-25T20:08:22.317 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.317 INFO:teuthology.orchestra.run.smithi135.stdout: osd open classes on start = true 2024-04-25T20:08:22.317 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug pg log writeout = true 2024-04-25T20:08:22.317 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.317 INFO:teuthology.orchestra.run.smithi135.stdout: osd deep scrub update digest min age = 30 2024-04-25T20:08:22.317 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.317 INFO:teuthology.orchestra.run.smithi135.stdout: osd map max advance = 10 2024-04-25T20:08:22.317 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.318 INFO:teuthology.orchestra.run.smithi135.stdout: journal zero on create = true 2024-04-25T20:08:22.318 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.318 INFO:teuthology.orchestra.run.smithi135.stdout: filestore ondisk finisher threads = 3 2024-04-25T20:08:22.318 INFO:teuthology.orchestra.run.smithi135.stdout: filestore apply finisher threads = 3 2024-04-25T20:08:22.318 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.318 INFO:teuthology.orchestra.run.smithi135.stdout: bdev debug aio = true 2024-04-25T20:08:22.318 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug misdirected ops = true 2024-04-25T20:08:22.318 INFO:teuthology.orchestra.run.smithi135.stdout: bluestore block size = 96636764160 2024-04-25T20:08:22.318 INFO:teuthology.orchestra.run.smithi135.stdout: bluestore compression algorithm = lz4 2024-04-25T20:08:22.318 INFO:teuthology.orchestra.run.smithi135.stdout: bluestore compression mode = aggressive 2024-04-25T20:08:22.318 INFO:teuthology.orchestra.run.smithi135.stdout: bluestore fsck on mount = True 2024-04-25T20:08:22.319 INFO:teuthology.orchestra.run.smithi135.stdout: debug bluefs = 1/20 2024-04-25T20:08:22.319 INFO:teuthology.orchestra.run.smithi135.stdout: debug bluestore = 1/20 2024-04-25T20:08:22.319 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-04-25T20:08:22.319 INFO:teuthology.orchestra.run.smithi135.stdout: debug osd = 20 2024-04-25T20:08:22.319 INFO:teuthology.orchestra.run.smithi135.stdout: debug rocksdb = 4/10 2024-04-25T20:08:22.319 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T20:08:22.319 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd full ratio = 0.9 2024-04-25T20:08:22.319 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd nearfull ratio = 0.8 2024-04-25T20:08:22.319 INFO:teuthology.orchestra.run.smithi135.stdout: osd blocked scrub grace period = 3600 2024-04-25T20:08:22.319 INFO:teuthology.orchestra.run.smithi135.stdout: osd failsafe full ratio = 0.95 2024-04-25T20:08:22.319 INFO:teuthology.orchestra.run.smithi135.stdout: osd max markdown count = 1000 2024-04-25T20:08:22.319 INFO:teuthology.orchestra.run.smithi135.stdout: osd objectstore = bluestore 2024-04-25T20:08:22.319 INFO:teuthology.orchestra.run.smithi135.stdout: osd shutdown pgref assert = True 2024-04-25T20:08:22.319 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.319 INFO:teuthology.orchestra.run.smithi135.stdout:[mgr] 2024-04-25T20:08:22.319 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-04-25T20:08:22.320 INFO:teuthology.orchestra.run.smithi135.stdout: debug mgr = 20 2024-04-25T20:08:22.320 INFO:teuthology.orchestra.run.smithi135.stdout: debug mon = 20 2024-04-25T20:08:22.320 INFO:teuthology.orchestra.run.smithi135.stdout: debug auth = 20 2024-04-25T20:08:22.320 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min pgs per osd = 4 2024-04-25T20:08:22.320 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min bytes per osd = 10 2024-04-25T20:08:22.320 INFO:teuthology.orchestra.run.smithi135.stdout: mgr/telemetry/nag = false 2024-04-25T20:08:22.320 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.320 INFO:teuthology.orchestra.run.smithi135.stdout:[mon] 2024-04-25T20:08:22.320 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-04-25T20:08:22.320 INFO:teuthology.orchestra.run.smithi135.stdout: debug mon = 20 2024-04-25T20:08:22.320 INFO:teuthology.orchestra.run.smithi135.stdout: debug paxos = 20 2024-04-25T20:08:22.320 INFO:teuthology.orchestra.run.smithi135.stdout: debug auth = 20 2024-04-25T20:08:22.320 INFO:teuthology.orchestra.run.smithi135.stdout: mon data avail warn = 5 2024-04-25T20:08:22.320 INFO:teuthology.orchestra.run.smithi135.stdout: mon mgr mkfs grace = 240 2024-04-25T20:08:22.321 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min pgs per osd = 4 2024-04-25T20:08:22.321 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd reporter subtree level = osd 2024-04-25T20:08:22.321 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd prime pg temp = true 2024-04-25T20:08:22.321 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min bytes per osd = 10 2024-04-25T20:08:22.321 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.321 INFO:teuthology.orchestra.run.smithi135.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T20:08:22.321 INFO:teuthology.orchestra.run.smithi135.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T20:08:22.321 INFO:teuthology.orchestra.run.smithi135.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T20:08:22.321 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.321 INFO:teuthology.orchestra.run.smithi135.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T20:08:22.321 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T20:08:22.321 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T20:08:22.321 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.321 INFO:teuthology.orchestra.run.smithi135.stdout: # 1m isn't quite enough 2024-04-25T20:08:22.321 INFO:teuthology.orchestra.run.smithi135.stdout: mon_down_mkfs_grace = 2m 2024-04-25T20:08:22.322 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.322 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_filestore_osds = false 2024-04-25T20:08:22.322 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-25T20:08:22.322 INFO:teuthology.orchestra.run.smithi135.stdout:[client] 2024-04-25T20:08:22.322 INFO:teuthology.orchestra.run.smithi135.stdout: rgw cache enabled = true 2024-04-25T20:08:22.322 INFO:teuthology.orchestra.run.smithi135.stdout: rgw enable ops log = true 2024-04-25T20:08:22.322 INFO:teuthology.orchestra.run.smithi135.stdout: rgw enable usage log = true 2024-04-25T20:08:22.322 INFO:teuthology.orchestra.run.smithi135.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T20:08:22.322 INFO:teuthology.orchestra.run.smithi135.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T20:08:22.322 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.a] 2024-04-25T20:08:22.322 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.c] 2024-04-25T20:08:22.322 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.b] 2024-04-25T20:08:22.324 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T20:08:22.324 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T20:08:22.417 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T20:08:22.417 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-04-25T20:08:22.417 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T20:08:22.433 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-04-25T20:08:22.433 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T20:08:22.488 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi039.front.sepia.ceph.com 2024-04-25T20:08:22.488 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-04-25T20:08:22.488 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T20:08:22.488 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T20:08:22.561 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-04-25T20:08:22.562 DEBUG:teuthology.orchestra.run.smithi039:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T20:08:22.617 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi135.front.sepia.ceph.com 2024-04-25T20:08:22.617 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-25T20:08:22.617 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T20:08:22.617 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T20:08:22.653 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-25T20:08:22.653 DEBUG:teuthology.orchestra.run.smithi135:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T20:08:22.708 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T20:08:22.708 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T20:08:22.709 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T20:08:22.759 INFO:teuthology.orchestra.run.smithi039.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T20:08:22.761 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T20:08:22.823 INFO:teuthology.orchestra.run.smithi135.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T20:08:22.826 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T20:08:22.826 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T20:08:22.826 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T20:08:22.866 INFO:teuthology.orchestra.run.smithi039.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T20:08:22.878 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T20:08:22.917 INFO:teuthology.orchestra.run.smithi135.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T20:08:22.929 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T20:08:22.929 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi039.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi135.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T20:08:22.929 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T20:08:22.953 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T20:08:22.953 INFO:tasks.ceph:role: osd.0 2024-04-25T20:08:22.953 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi039.front.sepia.ceph.com 2024-04-25T20:08:22.954 DEBUG:teuthology.orchestra.run.smithi039:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T20:08:23.046 INFO:teuthology.orchestra.run.smithi039.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T20:08:23.046 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T20:08:23.046 INFO:teuthology.orchestra.run.smithi039.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T20:08:23.046 INFO:teuthology.orchestra.run.smithi039.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T20:08:23.046 INFO:teuthology.orchestra.run.smithi039.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T20:08:23.046 INFO:teuthology.orchestra.run.smithi039.stdout: = sunit=0 swidth=0 blks 2024-04-25T20:08:23.046 INFO:teuthology.orchestra.run.smithi039.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T20:08:23.046 INFO:teuthology.orchestra.run.smithi039.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T20:08:23.046 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T20:08:23.046 INFO:teuthology.orchestra.run.smithi039.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T20:08:24.362 INFO:teuthology.orchestra.run.smithi039.stdout:Discarding blocks...Done. 2024-04-25T20:08:24.364 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi039.front.sepia.ceph.com -o noatime 2024-04-25T20:08:24.364 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T20:08:24.406 DEBUG:teuthology.orchestra.run.smithi039:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T20:08:24.478 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T20:08:24.545 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T20:08:24.545 INFO:tasks.ceph:role: osd.1 2024-04-25T20:08:24.545 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi039.front.sepia.ceph.com 2024-04-25T20:08:24.545 DEBUG:teuthology.orchestra.run.smithi039:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T20:08:24.639 INFO:teuthology.orchestra.run.smithi039.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T20:08:24.639 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T20:08:24.639 INFO:teuthology.orchestra.run.smithi039.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T20:08:24.639 INFO:teuthology.orchestra.run.smithi039.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T20:08:24.640 INFO:teuthology.orchestra.run.smithi039.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T20:08:24.640 INFO:teuthology.orchestra.run.smithi039.stdout: = sunit=0 swidth=0 blks 2024-04-25T20:08:24.640 INFO:teuthology.orchestra.run.smithi039.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T20:08:24.640 INFO:teuthology.orchestra.run.smithi039.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T20:08:24.640 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T20:08:24.640 INFO:teuthology.orchestra.run.smithi039.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T20:08:25.974 INFO:teuthology.orchestra.run.smithi039.stdout:Discarding blocks...Done. 2024-04-25T20:08:25.976 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi039.front.sepia.ceph.com -o noatime 2024-04-25T20:08:25.976 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T20:08:26.007 DEBUG:teuthology.orchestra.run.smithi039:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T20:08:26.077 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T20:08:26.141 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T20:08:26.141 INFO:tasks.ceph:role: osd.2 2024-04-25T20:08:26.141 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi039.front.sepia.ceph.com 2024-04-25T20:08:26.142 DEBUG:teuthology.orchestra.run.smithi039:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T20:08:26.233 INFO:teuthology.orchestra.run.smithi039.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T20:08:26.233 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T20:08:26.233 INFO:teuthology.orchestra.run.smithi039.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T20:08:26.233 INFO:teuthology.orchestra.run.smithi039.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T20:08:26.234 INFO:teuthology.orchestra.run.smithi039.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T20:08:26.234 INFO:teuthology.orchestra.run.smithi039.stdout: = sunit=0 swidth=0 blks 2024-04-25T20:08:26.234 INFO:teuthology.orchestra.run.smithi039.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T20:08:26.234 INFO:teuthology.orchestra.run.smithi039.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T20:08:26.234 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T20:08:26.234 INFO:teuthology.orchestra.run.smithi039.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T20:08:27.551 INFO:teuthology.orchestra.run.smithi039.stdout:Discarding blocks...Done. 2024-04-25T20:08:27.554 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi039.front.sepia.ceph.com -o noatime 2024-04-25T20:08:27.554 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T20:08:27.593 DEBUG:teuthology.orchestra.run.smithi039:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T20:08:27.662 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T20:08:27.730 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T20:08:27.730 INFO:tasks.ceph:role: osd.3 2024-04-25T20:08:27.730 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi039.front.sepia.ceph.com 2024-04-25T20:08:27.731 DEBUG:teuthology.orchestra.run.smithi039:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T20:08:27.824 INFO:teuthology.orchestra.run.smithi039.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T20:08:27.824 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T20:08:27.824 INFO:teuthology.orchestra.run.smithi039.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T20:08:27.825 INFO:teuthology.orchestra.run.smithi039.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T20:08:27.825 INFO:teuthology.orchestra.run.smithi039.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T20:08:27.825 INFO:teuthology.orchestra.run.smithi039.stdout: = sunit=0 swidth=0 blks 2024-04-25T20:08:27.825 INFO:teuthology.orchestra.run.smithi039.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T20:08:27.825 INFO:teuthology.orchestra.run.smithi039.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T20:08:27.825 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T20:08:27.825 INFO:teuthology.orchestra.run.smithi039.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T20:08:29.161 INFO:teuthology.orchestra.run.smithi039.stdout:Discarding blocks...Done. 2024-04-25T20:08:29.163 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi039.front.sepia.ceph.com -o noatime 2024-04-25T20:08:29.164 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T20:08:29.199 DEBUG:teuthology.orchestra.run.smithi039:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T20:08:29.271 DEBUG:teuthology.orchestra.run.smithi039:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T20:08:29.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-04-25T20:08:29.362+0000 7f20b6425640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T20:08:29.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-04-25T20:08:29.362+0000 7f20b6425640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T20:08:29.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-04-25T20:08:29.362+0000 7f20b6425640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T20:08:29.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-04-25T20:08:29.362+0000 7f20b6425640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T20:08:29.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-04-25T20:08:29.362+0000 7f20b6425640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T20:08:29.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-04-25T20:08:29.363+0000 7f20b6425640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T20:08:32.235 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T20:08:32.264 DEBUG:teuthology.orchestra.run.smithi039:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T20:08:32.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-04-25T20:08:32.353+0000 7fdfaf1fc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T20:08:32.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-04-25T20:08:32.354+0000 7fdfaf1fc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T20:08:32.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-04-25T20:08:32.354+0000 7fdfaf1fc640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T20:08:32.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-04-25T20:08:32.354+0000 7fdfaf1fc640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T20:08:32.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-04-25T20:08:32.354+0000 7fdfaf1fc640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T20:08:32.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-04-25T20:08:32.354+0000 7fdfaf1fc640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T20:08:35.211 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T20:08:35.241 DEBUG:teuthology.orchestra.run.smithi039:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T20:08:35.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-04-25T20:08:35.331+0000 7f86c5ea8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T20:08:35.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-04-25T20:08:35.332+0000 7f86c5ea8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T20:08:35.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-04-25T20:08:35.332+0000 7f86c5ea8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T20:08:35.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-04-25T20:08:35.332+0000 7f86c5ea8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T20:08:35.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-04-25T20:08:35.332+0000 7f86c5ea8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T20:08:35.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-04-25T20:08:35.333+0000 7f86c5ea8640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T20:08:38.196 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T20:08:38.225 DEBUG:teuthology.orchestra.run.smithi039:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T20:08:38.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-04-25T20:08:38.313+0000 7f4fc8a17640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T20:08:38.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-04-25T20:08:38.314+0000 7f4fc8a17640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T20:08:38.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-04-25T20:08:38.314+0000 7f4fc8a17640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T20:08:38.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-04-25T20:08:38.314+0000 7f4fc8a17640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T20:08:38.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-04-25T20:08:38.314+0000 7f4fc8a17640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T20:08:38.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-04-25T20:08:38.314+0000 7f4fc8a17640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T20:08:41.185 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T20:08:41.213 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T20:08:41.242 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T20:08:41.242 INFO:tasks.ceph:role: osd.4 2024-04-25T20:08:41.243 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi135.front.sepia.ceph.com 2024-04-25T20:08:41.243 DEBUG:teuthology.orchestra.run.smithi135:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T20:08:41.317 INFO:teuthology.orchestra.run.smithi135.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T20:08:41.317 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T20:08:41.317 INFO:teuthology.orchestra.run.smithi135.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T20:08:41.317 INFO:teuthology.orchestra.run.smithi135.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T20:08:41.317 INFO:teuthology.orchestra.run.smithi135.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T20:08:41.317 INFO:teuthology.orchestra.run.smithi135.stdout: = sunit=0 swidth=0 blks 2024-04-25T20:08:41.317 INFO:teuthology.orchestra.run.smithi135.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T20:08:41.318 INFO:teuthology.orchestra.run.smithi135.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T20:08:41.318 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T20:08:41.318 INFO:teuthology.orchestra.run.smithi135.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T20:08:41.815 INFO:teuthology.orchestra.run.smithi135.stdout:Discarding blocks...Done. 2024-04-25T20:08:41.817 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi135.front.sepia.ceph.com -o noatime 2024-04-25T20:08:41.818 DEBUG:teuthology.orchestra.run.smithi135:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T20:08:41.927 DEBUG:teuthology.orchestra.run.smithi135:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T20:08:41.964 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T20:08:42.031 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T20:08:42.031 INFO:tasks.ceph:role: osd.5 2024-04-25T20:08:42.031 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi135.front.sepia.ceph.com 2024-04-25T20:08:42.031 DEBUG:teuthology.orchestra.run.smithi135:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T20:08:42.104 INFO:teuthology.orchestra.run.smithi135.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T20:08:42.104 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T20:08:42.104 INFO:teuthology.orchestra.run.smithi135.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T20:08:42.104 INFO:teuthology.orchestra.run.smithi135.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T20:08:42.104 INFO:teuthology.orchestra.run.smithi135.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T20:08:42.104 INFO:teuthology.orchestra.run.smithi135.stdout: = sunit=0 swidth=0 blks 2024-04-25T20:08:42.105 INFO:teuthology.orchestra.run.smithi135.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T20:08:42.105 INFO:teuthology.orchestra.run.smithi135.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T20:08:42.105 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T20:08:42.105 INFO:teuthology.orchestra.run.smithi135.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T20:08:42.571 INFO:teuthology.orchestra.run.smithi135.stdout:Discarding blocks...Done. 2024-04-25T20:08:42.573 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi135.front.sepia.ceph.com -o noatime 2024-04-25T20:08:42.573 DEBUG:teuthology.orchestra.run.smithi135:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T20:08:42.669 DEBUG:teuthology.orchestra.run.smithi135:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T20:08:42.695 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T20:08:42.760 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T20:08:42.760 INFO:tasks.ceph:role: osd.6 2024-04-25T20:08:42.760 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi135.front.sepia.ceph.com 2024-04-25T20:08:42.760 DEBUG:teuthology.orchestra.run.smithi135:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T20:08:42.830 INFO:teuthology.orchestra.run.smithi135.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T20:08:42.830 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T20:08:42.830 INFO:teuthology.orchestra.run.smithi135.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T20:08:42.830 INFO:teuthology.orchestra.run.smithi135.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T20:08:42.830 INFO:teuthology.orchestra.run.smithi135.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T20:08:42.830 INFO:teuthology.orchestra.run.smithi135.stdout: = sunit=0 swidth=0 blks 2024-04-25T20:08:42.831 INFO:teuthology.orchestra.run.smithi135.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T20:08:42.831 INFO:teuthology.orchestra.run.smithi135.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T20:08:42.831 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T20:08:42.831 INFO:teuthology.orchestra.run.smithi135.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T20:08:43.314 INFO:teuthology.orchestra.run.smithi135.stdout:Discarding blocks...Done. 2024-04-25T20:08:43.317 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi135.front.sepia.ceph.com -o noatime 2024-04-25T20:08:43.317 DEBUG:teuthology.orchestra.run.smithi135:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T20:08:43.423 DEBUG:teuthology.orchestra.run.smithi135:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T20:08:43.452 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T20:08:43.515 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T20:08:43.515 INFO:tasks.ceph:role: osd.7 2024-04-25T20:08:43.516 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi135.front.sepia.ceph.com 2024-04-25T20:08:43.516 DEBUG:teuthology.orchestra.run.smithi135:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T20:08:43.586 INFO:teuthology.orchestra.run.smithi135.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T20:08:43.587 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T20:08:43.587 INFO:teuthology.orchestra.run.smithi135.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T20:08:43.587 INFO:teuthology.orchestra.run.smithi135.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T20:08:43.587 INFO:teuthology.orchestra.run.smithi135.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T20:08:43.587 INFO:teuthology.orchestra.run.smithi135.stdout: = sunit=0 swidth=0 blks 2024-04-25T20:08:43.587 INFO:teuthology.orchestra.run.smithi135.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T20:08:43.587 INFO:teuthology.orchestra.run.smithi135.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T20:08:43.587 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T20:08:43.587 INFO:teuthology.orchestra.run.smithi135.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T20:08:44.108 INFO:teuthology.orchestra.run.smithi135.stdout:Discarding blocks...Done. 2024-04-25T20:08:44.111 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi135.front.sepia.ceph.com -o noatime 2024-04-25T20:08:44.111 DEBUG:teuthology.orchestra.run.smithi135:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T20:08:44.238 DEBUG:teuthology.orchestra.run.smithi135:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T20:08:44.269 DEBUG:teuthology.orchestra.run.smithi135:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T20:08:44.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-25T20:08:44.365+0000 7f6cdc61f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T20:08:44.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-25T20:08:44.365+0000 7f6cdc61f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T20:08:44.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-25T20:08:44.365+0000 7f6cdc61f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T20:08:44.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-25T20:08:44.365+0000 7f6cdc61f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T20:08:44.366 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-25T20:08:44.365+0000 7f6cdc61f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T20:08:44.366 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-25T20:08:44.365+0000 7f6cdc61f640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T20:08:47.216 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T20:08:47.245 DEBUG:teuthology.orchestra.run.smithi135:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T20:08:47.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-25T20:08:47.334+0000 7f73fe6b1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T20:08:47.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-25T20:08:47.334+0000 7f73fe6b1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T20:08:47.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-25T20:08:47.334+0000 7f73fe6b1640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T20:08:47.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-25T20:08:47.334+0000 7f73fe6b1640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T20:08:47.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-25T20:08:47.334+0000 7f73fe6b1640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T20:08:47.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-25T20:08:47.334+0000 7f73fe6b1640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T20:08:50.194 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T20:08:50.222 DEBUG:teuthology.orchestra.run.smithi135:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T20:08:50.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-25T20:08:50.309+0000 7f91edfb2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T20:08:50.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-25T20:08:50.310+0000 7f91edfb2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T20:08:50.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-25T20:08:50.310+0000 7f91edfb2640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T20:08:50.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-25T20:08:50.310+0000 7f91edfb2640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T20:08:50.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-25T20:08:50.310+0000 7f91edfb2640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T20:08:50.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-25T20:08:50.310+0000 7f91edfb2640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T20:08:53.166 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T20:08:53.194 DEBUG:teuthology.orchestra.run.smithi135:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T20:08:53.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-25T20:08:53.282+0000 7fef953d6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T20:08:53.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-25T20:08:53.282+0000 7fef953d6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T20:08:53.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-25T20:08:53.282+0000 7fef953d6640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T20:08:53.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-25T20:08:53.282+0000 7fef953d6640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T20:08:53.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-25T20:08:53.282+0000 7fef953d6640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T20:08:53.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-25T20:08:53.283+0000 7fef953d6640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T20:08:56.138 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T20:08:56.166 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T20:08:56.166 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-04-25T20:08:56.166 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T20:08:56.195 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-04-25T20:08:56.196 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T20:08:56.263 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-04-25T20:08:56.264 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T20:08:56.330 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-04-25T20:08:56.330 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T20:08:56.396 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-04-25T20:08:56.396 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T20:08:56.462 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-25T20:08:56.463 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T20:08:56.491 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-25T20:08:56.491 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T20:08:56.557 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-25T20:08:56.558 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T20:08:56.624 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-25T20:08:56.624 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T20:08:56.690 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-25T20:08:56.690 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T20:08:56.755 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-04-25T20:08:56.755 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T20:08:56.770 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-25T20:08:56.771 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T20:08:56.810 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T20:08:56.811 DEBUG:teuthology.orchestra.run.smithi039:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T20:08:56.813 DEBUG:teuthology.orchestra.run.smithi135:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T20:08:56.855 INFO:teuthology.orchestra.run.smithi039.stdout:[mgr.y] 2024-04-25T20:08:56.855 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQA2uCpmUjJKLRAAEoGaJeJK1QoZFE3fPc38dA== 2024-04-25T20:08:56.855 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.0] 2024-04-25T20:08:56.855 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQA9uCpm1KmjFRAAvMGyLBnRFmoNk/1yrgwEEA== 2024-04-25T20:08:56.855 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.1] 2024-04-25T20:08:56.855 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQBAuCpmPfMiFRAA9xndjLgkFF26WbuWmM9Rrg== 2024-04-25T20:08:56.855 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.2] 2024-04-25T20:08:56.855 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQBDuCpmV7PWExAAkF82JNX4I0mPD2pngvIYjQ== 2024-04-25T20:08:56.855 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.3] 2024-04-25T20:08:56.855 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQBGuCpmPtvQEhAA3wGs3sRLtIh7QgUqwcdDVw== 2024-04-25T20:08:56.855 INFO:teuthology.orchestra.run.smithi039.stdout:[mgr.x] 2024-04-25T20:08:56.856 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQA2uCpmVEweMRAAfxfQauEqa5N2ay/1W9H7mQ== 2024-04-25T20:08:56.856 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.4] 2024-04-25T20:08:56.856 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQBMuCpmL+HJFRAAmQUHpnhPddvFJRXxaWed3g== 2024-04-25T20:08:56.856 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.5] 2024-04-25T20:08:56.856 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQBPuCpm8mnwExAANgZuL1yxsGJq++HTIwxexA== 2024-04-25T20:08:56.856 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.6] 2024-04-25T20:08:56.856 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQBSuCpm8oJ+EhAAtZwiIFG8AaUbJOAQ3UqJ4w== 2024-04-25T20:08:56.856 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.7] 2024-04-25T20:08:56.856 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQBVuCpmpN/cEBAArk6UKxEGGJkHWHznw744fw== 2024-04-25T20:08:56.856 INFO:teuthology.orchestra.run.smithi039.stdout:[client.0] 2024-04-25T20:08:56.856 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQA2uCpmHfqoMxAAmiwG9nlXRtB/zsrSyoYWhQ== 2024-04-25T20:08:56.856 INFO:teuthology.orchestra.run.smithi039.stdout:[client.1] 2024-04-25T20:08:56.857 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQA2uCpm/WG3NhAAnSAsDABI3sbX0InjpNqyZQ== 2024-04-25T20:08:56.873 INFO:teuthology.orchestra.run.smithi135.stdout:[mgr.y] 2024-04-25T20:08:56.873 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQA2uCpmUjJKLRAAEoGaJeJK1QoZFE3fPc38dA== 2024-04-25T20:08:56.874 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.0] 2024-04-25T20:08:56.874 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQA9uCpm1KmjFRAAvMGyLBnRFmoNk/1yrgwEEA== 2024-04-25T20:08:56.874 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.1] 2024-04-25T20:08:56.874 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBAuCpmPfMiFRAA9xndjLgkFF26WbuWmM9Rrg== 2024-04-25T20:08:56.874 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.2] 2024-04-25T20:08:56.874 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBDuCpmV7PWExAAkF82JNX4I0mPD2pngvIYjQ== 2024-04-25T20:08:56.874 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.3] 2024-04-25T20:08:56.874 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBGuCpmPtvQEhAA3wGs3sRLtIh7QgUqwcdDVw== 2024-04-25T20:08:56.874 INFO:teuthology.orchestra.run.smithi135.stdout:[mgr.x] 2024-04-25T20:08:56.874 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQA2uCpmVEweMRAAfxfQauEqa5N2ay/1W9H7mQ== 2024-04-25T20:08:56.874 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.4] 2024-04-25T20:08:56.874 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBMuCpmL+HJFRAAmQUHpnhPddvFJRXxaWed3g== 2024-04-25T20:08:56.875 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.5] 2024-04-25T20:08:56.875 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBPuCpm8mnwExAANgZuL1yxsGJq++HTIwxexA== 2024-04-25T20:08:56.875 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.6] 2024-04-25T20:08:56.875 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBSuCpm8oJ+EhAAtZwiIFG8AaUbJOAQ3UqJ4w== 2024-04-25T20:08:56.875 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.7] 2024-04-25T20:08:56.875 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBVuCpmpN/cEBAArk6UKxEGGJkHWHznw744fw== 2024-04-25T20:08:56.875 INFO:teuthology.orchestra.run.smithi135.stdout:[client.0] 2024-04-25T20:08:56.875 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQA2uCpmHfqoMxAAmiwG9nlXRtB/zsrSyoYWhQ== 2024-04-25T20:08:56.875 INFO:teuthology.orchestra.run.smithi135.stdout:[client.1] 2024-04-25T20:08:56.875 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQA2uCpm/WG3NhAAnSAsDABI3sbX0InjpNqyZQ== 2024-04-25T20:08:56.876 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T20:08:56.900 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T20:08:56.967 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T20:08:56.969 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T20:08:57.017 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T20:08:57.020 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T20:08:57.068 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T20:08:57.070 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T20:08:57.119 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T20:08:57.121 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T20:08:57.169 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T20:08:57.171 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T20:08:57.219 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T20:08:57.221 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T20:08:57.269 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T20:08:57.272 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T20:08:57.320 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T20:08:57.322 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T20:08:57.370 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T20:08:57.372 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T20:08:57.420 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T20:08:57.422 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T20:08:57.474 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T20:08:57.476 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T20:08:57.527 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T20:08:57.528 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T20:08:57.551 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T20:08:57.655 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T20:08:57.679 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T20:08:57.743 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T20:08:57.843 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T20:08:57.868 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T20:08:57.892 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T20:08:57.986 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T20:08:58.009 DEBUG:teuthology.orchestra.run.smithi039:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T20:08:58.012 DEBUG:teuthology.orchestra.run.smithi135:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T20:08:58.064 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T20:08:58.065 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T20:08:58.065 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T20:08:58.067 INFO:tasks.ceph.mon.a:Started 2024-04-25T20:08:58.067 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T20:08:58.067 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T20:08:58.069 INFO:tasks.ceph.mon.c:Started 2024-04-25T20:08:58.069 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T20:08:58.069 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T20:08:58.107 INFO:tasks.ceph.mon.b:Started 2024-04-25T20:08:58.107 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T20:08:58.108 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T20:08:58.108 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T20:08:58.110 INFO:tasks.ceph.mgr.y:Started 2024-04-25T20:08:58.111 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T20:08:58.111 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T20:08:58.113 INFO:tasks.ceph.mgr.x:Started 2024-04-25T20:08:58.114 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T20:08:58.114 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T20:09:03.484 INFO:teuthology.orchestra.run.smithi039.stderr:adjusted tunables profile to default 2024-04-25T20:09:03.496 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T20:09:03.496 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T20:09:03.497 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-04-25T20:09:03.497 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T20:09:03.514 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-04-25T20:09:03.514+0000 7f6d3c614040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T20:09:03.521 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-04-25T20:09:03.521 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T20:09:03.585 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-04-25T20:09:03.586 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T20:09:03.650 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-04-25T20:09:03.651 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T20:09:03.670 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-04-25T20:09:03.670+0000 7f6d3c614040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:09:03.694 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-25T20:09:03.694 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T20:09:03.723 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-25T20:09:03.723 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T20:09:03.727 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-04-25T20:09:03.727+0000 7f6d3c614040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T20:09:03.791 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-25T20:09:03.791 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T20:09:03.857 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-25T20:09:03.857 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T20:09:03.922 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd new 7a7118ba-4aee-404f-97e3-94c50e2996c4 0 2024-04-25T20:09:03.982 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-04-25T20:09:03.982+0000 7f6d3c614040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T20:09:04.029 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-04-25T20:09:04.029+0000 7f6d3c614040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T20:09:04.146 INFO:teuthology.orchestra.run.smithi135.stdout:0 2024-04-25T20:09:04.157 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd new 13637dbe-e37f-457f-b68b-8eed1b970617 1 2024-04-25T20:09:04.285 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-04-25T20:09:04.284+0000 7fe8c064b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T20:09:04.344 INFO:teuthology.orchestra.run.smithi135.stdout:1 2024-04-25T20:09:04.355 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd new 29e20ce2-4776-455d-8cf4-d539e738a6a7 2 2024-04-25T20:09:04.437 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-04-25T20:09:04.436+0000 7fe8c064b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:09:04.494 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-04-25T20:09:04.492+0000 7fe8c064b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T20:09:04.544 INFO:teuthology.orchestra.run.smithi135.stdout:2 2024-04-25T20:09:04.556 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd new a8b34e02-0898-4303-9fd5-62cbba9fec5c 3 2024-04-25T20:09:04.740 INFO:teuthology.orchestra.run.smithi135.stdout:3 2024-04-25T20:09:04.751 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd new e92b5c67-6908-4fd4-9940-5cb98b5ee847 4 2024-04-25T20:09:04.752 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-04-25T20:09:04.751+0000 7fe8c064b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T20:09:04.753 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-04-25T20:09:04.753+0000 7f6d3c614040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T20:09:04.800 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-04-25T20:09:04.799+0000 7fe8c064b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T20:09:04.843 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-04-25T20:09:04.842+0000 7f6d3c614040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T20:09:04.887 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-04-25T20:09:04.887+0000 7f6d3c614040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T20:09:04.939 INFO:teuthology.orchestra.run.smithi135.stdout:4 2024-04-25T20:09:04.951 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd new d74d8fef-0211-413b-bff2-f97776fc1faa 5 2024-04-25T20:09:05.058 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-04-25T20:09:05.058+0000 7f6d3c614040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T20:09:05.112 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-04-25T20:09:05.112+0000 7f6d3c614040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T20:09:05.136 INFO:teuthology.orchestra.run.smithi135.stdout:5 2024-04-25T20:09:05.148 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd new 7dc84c6e-76d8-4f22-8b17-51f62765762d 6 2024-04-25T20:09:05.167 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-04-25T20:09:05.167+0000 7f6d3c614040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T20:09:05.283 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-04-25T20:09:05.283+0000 7f6d3c614040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:09:05.341 INFO:teuthology.orchestra.run.smithi135.stdout:6 2024-04-25T20:09:05.353 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd new 7adb3187-aed3-48af-868c-7230e9bfc85e 7 2024-04-25T20:09:05.526 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-04-25T20:09:05.525+0000 7fe8c064b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T20:09:05.539 INFO:teuthology.orchestra.run.smithi135.stdout:7 2024-04-25T20:09:05.551 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T20:09:05.551 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T20:09:05.553 INFO:tasks.ceph.osd.0:Started 2024-04-25T20:09:05.554 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T20:09:05.554 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T20:09:05.556 INFO:tasks.ceph.osd.1:Started 2024-04-25T20:09:05.556 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T20:09:05.556 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T20:09:05.558 INFO:tasks.ceph.osd.2:Started 2024-04-25T20:09:05.558 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T20:09:05.558 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T20:09:05.560 INFO:tasks.ceph.osd.3:Started 2024-04-25T20:09:05.561 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T20:09:05.561 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T20:09:05.563 INFO:tasks.ceph.osd.4:Started 2024-04-25T20:09:05.563 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T20:09:05.563 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T20:09:05.565 INFO:tasks.ceph.osd.5:Started 2024-04-25T20:09:05.565 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T20:09:05.565 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T20:09:05.567 INFO:tasks.ceph.osd.6:Started 2024-04-25T20:09:05.567 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T20:09:05.567 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T20:09:05.569 INFO:tasks.ceph.osd.7:Started 2024-04-25T20:09:05.570 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T20:09:05.624 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-04-25T20:09:05.622+0000 7fe8c064b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T20:09:05.672 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-04-25T20:09:05.671+0000 7fe8c064b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T20:09:05.717 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-04-25T20:09:05.716+0000 7f6d3c614040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T20:09:05.765 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:09:05.766 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":10,"fsid":"975ceafd-555d-4f71-81e4-1f70364da00f","created":"2024-04-25T20:09:03.377320+0000","modified":"2024-04-25T20:09:05.529264+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T20:09:05.529264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7a7118ba-4aee-404f-97e3-94c50e2996c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"13637dbe-e37f-457f-b68b-8eed1b970617","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"29e20ce2-4776-455d-8cf4-d539e738a6a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a8b34e02-0898-4303-9fd5-62cbba9fec5c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e92b5c67-6908-4fd4-9940-5cb98b5ee847","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d74d8fef-0211-413b-bff2-f97776fc1faa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7dc84c6e-76d8-4f22-8b17-51f62765762d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"7adb3187-aed3-48af-868c-7230e9bfc85e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T20:09:05.777 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T20:09:05.777 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T20:09:05.837 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-04-25T20:09:05.836+0000 7fe8c064b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T20:09:05.891 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-04-25T20:09:05.890+0000 7fe8c064b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T20:09:05.944 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-04-25T20:09:05.943+0000 7fe8c064b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T20:09:05.984 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-04-25T20:09:05.984+0000 7f6d3c614040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T20:09:06.038 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-04-25T20:09:06.038+0000 7f6d3c614040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T20:09:06.056 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-04-25T20:09:06.055+0000 7fe8c064b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:09:06.158 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-04-25T20:09:06.157+0000 7f6d3c614040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T20:09:06.163 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:06.162+0000 7f34ff40e640 -1 Falling back to public interface 2024-04-25T20:09:06.178 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:06.177+0000 7ffa5c22e640 -1 Falling back to public interface 2024-04-25T20:09:06.178 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:06.178+0000 7f2fe6815640 -1 Falling back to public interface 2024-04-25T20:09:06.179 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:06.178+0000 7fa97a799640 -1 Falling back to public interface 2024-04-25T20:09:06.182 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:06.181+0000 7f6213144640 -1 Falling back to public interface 2024-04-25T20:09:06.182 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:06.181+0000 7fecb4a10640 -1 Falling back to public interface 2024-04-25T20:09:06.193 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:06.192+0000 7f6f2e01c640 -1 Falling back to public interface 2024-04-25T20:09:06.194 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:06.193+0000 7ffac3714640 -1 Falling back to public interface 2024-04-25T20:09:06.195 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:06.195+0000 7ffac26bd640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6825/0: (98) Address already in use 2024-04-25T20:09:06.437 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-04-25T20:09:06.436+0000 7fe8c064b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T20:09:06.449 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-04-25T20:09:06.448+0000 7f6d3c614040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T20:09:06.498 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-04-25T20:09:06.498+0000 7f6d3c614040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T20:09:06.582 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T20:09:06.661 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-04-25T20:09:06.661+0000 7f6d3c614040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T20:09:06.691 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-04-25T20:09:06.690+0000 7fe8c064b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T20:09:06.744 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-04-25T20:09:06.743+0000 7fe8c064b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T20:09:06.746 INFO:teuthology.misc.health.smithi039.stdout: 2024-04-25T20:09:06.746 INFO:teuthology.misc.health.smithi039.stdout:{"epoch":10,"fsid":"975ceafd-555d-4f71-81e4-1f70364da00f","created":"2024-04-25T20:09:03.377320+0000","modified":"2024-04-25T20:09:05.529264+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T20:09:05.529264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7a7118ba-4aee-404f-97e3-94c50e2996c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"13637dbe-e37f-457f-b68b-8eed1b970617","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"29e20ce2-4776-455d-8cf4-d539e738a6a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a8b34e02-0898-4303-9fd5-62cbba9fec5c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e92b5c67-6908-4fd4-9940-5cb98b5ee847","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d74d8fef-0211-413b-bff2-f97776fc1faa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7dc84c6e-76d8-4f22-8b17-51f62765762d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"7adb3187-aed3-48af-868c-7230e9bfc85e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T20:09:06.756 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T20:09:06.808 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-04-25T20:09:06.807+0000 7f6d3c614040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T20:09:06.858 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-04-25T20:09:06.858+0000 7f6d3c614040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T20:09:06.860 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-04-25T20:09:06.859+0000 7fe8c064b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T20:09:06.931 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-04-25T20:09:06.930+0000 7f6d3c614040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T20:09:06.978 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-04-25T20:09:06.977+0000 7f6d3c614040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T20:09:07.139 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-04-25T20:09:07.138+0000 7fe8c064b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T20:09:07.186 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-04-25T20:09:07.185+0000 7fe8c064b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T20:09:07.344 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-04-25T20:09:07.343+0000 7fe8c064b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T20:09:07.470 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-04-25T20:09:07.469+0000 7f6d3c614040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T20:09:07.480 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-04-25T20:09:07.479+0000 7fe8c064b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T20:09:07.529 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-04-25T20:09:07.528+0000 7fe8c064b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T20:09:07.589 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-04-25T20:09:07.588+0000 7f6d3c614040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T20:09:07.601 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-04-25T20:09:07.600+0000 7fe8c064b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T20:09:07.648 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-04-25T20:09:07.647+0000 7fe8c064b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T20:09:08.117 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-04-25T20:09:08.116+0000 7fe8c064b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T20:09:08.234 INFO:tasks.ceph.mgr.x.smithi135.stderr:2024-04-25T20:09:08.233+0000 7fe8c064b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T20:09:08.608 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-04-25T20:09:08.608+0000 7f6cdcf99640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T20:09:08.609 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-04-25T20:09:08.609+0000 7f6cdcf99640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T20:09:09.661 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:09.661+0000 7f6213144640 -1 osd.0 0 log_to_monitors true 2024-04-25T20:09:09.712 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:09.711+0000 7f2fe6815640 -1 osd.1 0 log_to_monitors true 2024-04-25T20:09:09.719 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:09.719+0000 7fecb4a10640 -1 osd.3 0 log_to_monitors true 2024-04-25T20:09:09.719 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:09.719+0000 7ffa5c22e640 -1 osd.2 0 log_to_monitors true 2024-04-25T20:09:09.731 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:09.731+0000 7fa97a799640 -1 osd.6 0 log_to_monitors true 2024-04-25T20:09:09.733 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:09.733+0000 7f34ff40e640 -1 osd.4 0 log_to_monitors true 2024-04-25T20:09:09.780 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:09.779+0000 7f6f2e01c640 -1 osd.7 0 log_to_monitors true 2024-04-25T20:09:09.780 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:09.779+0000 7ffac3714640 -1 osd.5 0 log_to_monitors true 2024-04-25T20:09:12.718 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:12.717+0000 7f6f29959640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T20:09:12.723 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:12.722+0000 7ffabf6a1640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T20:09:12.734 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:12.733+0000 7f6f2177a640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:09:12.734 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:12.733+0000 7f34facc6640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T20:09:12.741 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:12.741+0000 7ffab74c2640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:09:12.745 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:12.745+0000 7f34f2ae7640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:09:12.750 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:12.750+0000 7ffa577af640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T20:09:12.751 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:12.751+0000 7fa976f3d640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T20:09:12.765 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:12.764+0000 7fa96e547640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:09:12.766 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:12.765+0000 7f2fe21a8640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T20:09:12.767 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:12.766+0000 7ffa4f5d0640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:09:12.781 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:12.779+0000 7f2fd9fc9640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:09:12.785 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:12.785+0000 7f620f0d1640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T20:09:12.801 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:12.801+0000 7f6206ef2640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:09:12.811 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:12.811+0000 7fecb0448640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T20:09:12.821 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:12.821+0000 7feca8269640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:09:13.563 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T20:09:13.868 INFO:teuthology.misc.health.smithi039.stdout: 2024-04-25T20:09:13.868 INFO:teuthology.misc.health.smithi039.stdout:{"epoch":13,"fsid":"975ceafd-555d-4f71-81e4-1f70364da00f","created":"2024-04-25T20:09:03.377320+0000","modified":"2024-04-25T20:09:13.740341+0000","last_up_change":"2024-04-25T20:09:13.740341+0000","last_in_change":"2024-04-25T20:09:05.529264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7a7118ba-4aee-404f-97e3-94c50e2996c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":31832},{"type":"v1","addr":"172.21.15.39:6818","nonce":31832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":31832},{"type":"v1","addr":"172.21.15.39:6822","nonce":31832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":31832},{"type":"v1","addr":"172.21.15.39:6830","nonce":31832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":31832},{"type":"v1","addr":"172.21.15.39:6826","nonce":31832}]},"public_addr":"172.21.15.39:6818/31832","cluster_addr":"172.21.15.39:6822/31832","heartbeat_back_addr":"172.21.15.39:6830/31832","heartbeat_front_addr":"172.21.15.39:6826/31832","state":["exists","up"]},{"osd":1,"uuid":"13637dbe-e37f-457f-b68b-8eed1b970617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":31831},{"type":"v1","addr":"172.21.15.39:6809","nonce":31831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":31831},{"type":"v1","addr":"172.21.15.39:6811","nonce":31831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":31831},{"type":"v1","addr":"172.21.15.39:6815","nonce":31831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":31831},{"type":"v1","addr":"172.21.15.39:6813","nonce":31831}]},"public_addr":"172.21.15.39:6809/31831","cluster_addr":"172.21.15.39:6811/31831","heartbeat_back_addr":"172.21.15.39:6815/31831","heartbeat_front_addr":"172.21.15.39:6813/31831","state":["exists","up"]},{"osd":2,"uuid":"29e20ce2-4776-455d-8cf4-d539e738a6a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":31829},{"type":"v1","addr":"172.21.15.39:6801","nonce":31829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":31829},{"type":"v1","addr":"172.21.15.39:6803","nonce":31829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":31829},{"type":"v1","addr":"172.21.15.39:6807","nonce":31829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":31829},{"type":"v1","addr":"172.21.15.39:6805","nonce":31829}]},"public_addr":"172.21.15.39:6801/31829","cluster_addr":"172.21.15.39:6803/31829","heartbeat_back_addr":"172.21.15.39:6807/31829","heartbeat_front_addr":"172.21.15.39:6805/31829","state":["exists","up"]},{"osd":3,"uuid":"a8b34e02-0898-4303-9fd5-62cbba9fec5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6817","nonce":31830},{"type":"v1","addr":"172.21.15.39:6819","nonce":31830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6821","nonce":31830},{"type":"v1","addr":"172.21.15.39:6823","nonce":31830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6829","nonce":31830},{"type":"v1","addr":"172.21.15.39:6831","nonce":31830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6825","nonce":31830},{"type":"v1","addr":"172.21.15.39:6827","nonce":31830}]},"public_addr":"172.21.15.39:6819/31830","cluster_addr":"172.21.15.39:6823/31830","heartbeat_back_addr":"172.21.15.39:6831/31830","heartbeat_front_addr":"172.21.15.39:6827/31830","state":["exists","up"]},{"osd":4,"uuid":"e92b5c67-6908-4fd4-9940-5cb98b5ee847","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":31724},{"type":"v1","addr":"172.21.15.135:6801","nonce":31724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":31724},{"type":"v1","addr":"172.21.15.135:6803","nonce":31724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":31724},{"type":"v1","addr":"172.21.15.135:6807","nonce":31724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":31724},{"type":"v1","addr":"172.21.15.135:6805","nonce":31724}]},"public_addr":"172.21.15.135:6801/31724","cluster_addr":"172.21.15.135:6803/31724","heartbeat_back_addr":"172.21.15.135:6807/31724","heartbeat_front_addr":"172.21.15.135:6805/31724","state":["exists","up"]},{"osd":5,"uuid":"d74d8fef-0211-413b-bff2-f97776fc1faa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":31734},{"type":"v1","addr":"172.21.15.135:6822","nonce":31734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":31734},{"type":"v1","addr":"172.21.15.135:6826","nonce":31734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":31734},{"type":"v1","addr":"172.21.15.135:6831","nonce":31734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":31734},{"type":"v1","addr":"172.21.15.135:6829","nonce":31734}]},"public_addr":"172.21.15.135:6822/31734","cluster_addr":"172.21.15.135:6826/31734","heartbeat_back_addr":"172.21.15.135:6831/31734","heartbeat_front_addr":"172.21.15.135:6829/31734","state":["exists","up"]},{"osd":6,"uuid":"7dc84c6e-76d8-4f22-8b17-51f62765762d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":31725},{"type":"v1","addr":"172.21.15.135:6809","nonce":31725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":31725},{"type":"v1","addr":"172.21.15.135:6811","nonce":31725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":31725},{"type":"v1","addr":"172.21.15.135:6815","nonce":31725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":31725},{"type":"v1","addr":"172.21.15.135:6813","nonce":31725}]},"public_addr":"172.21.15.135:6809/31725","cluster_addr":"172.21.15.135:6811/31725","heartbeat_back_addr":"172.21.15.135:6815/31725","heartbeat_front_addr":"172.21.15.135:6813/31725","state":["exists","up"]},{"osd":7,"uuid":"7adb3187-aed3-48af-868c-7230e9bfc85e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":31737},{"type":"v1","addr":"172.21.15.135:6817","nonce":31737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":31737},{"type":"v1","addr":"172.21.15.135:6819","nonce":31737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":31737},{"type":"v1","addr":"172.21.15.135:6827","nonce":31737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":31737},{"type":"v1","addr":"172.21.15.135:6823","nonce":31737}]},"public_addr":"172.21.15.135:6817/31737","cluster_addr":"172.21.15.135:6819/31737","heartbeat_back_addr":"172.21.15.135:6827/31737","heartbeat_front_addr":"172.21.15.135:6823/31737","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T20:09:13.880 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T20:09:13.880 INFO:tasks.ceph:Creating RBD pool 2024-04-25T20:09:13.880 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph osd pool create rbd 8 2024-04-25T20:09:14.748 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'rbd' created 2024-04-25T20:09:14.763 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-25T20:09:15.763 INFO:teuthology.orchestra.run.smithi039.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-25T20:09:15.777 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T20:09:15.778 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T20:09:15.778 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T20:09:15.778 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T20:09:15.779 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T20:09:16.126 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:09:16.137 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":4,"flags":0,"active_gid":4103,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":31603},{"type":"v1","addr":"172.21.15.39:6833","nonce":31603}]},"active_addr":"172.21.15.39:6833/31603","active_change":"2024-04-25T20:09:07.591636+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4120,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":3365938615}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":1799559220}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":3402528347}]}]} 2024-04-25T20:09:16.142 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T20:09:16.142 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T20:09:16.143 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T20:09:16.436 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:09:16.436 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":15,"fsid":"975ceafd-555d-4f71-81e4-1f70364da00f","created":"2024-04-25T20:09:03.377320+0000","modified":"2024-04-25T20:09:15.745353+0000","last_up_change":"2024-04-25T20:09:13.740341+0000","last_in_change":"2024-04-25T20:09:05.529264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-25T20:09:14.202057+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-04-25T20:09:15.645688+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7a7118ba-4aee-404f-97e3-94c50e2996c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":31832},{"type":"v1","addr":"172.21.15.39:6818","nonce":31832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":31832},{"type":"v1","addr":"172.21.15.39:6822","nonce":31832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":31832},{"type":"v1","addr":"172.21.15.39:6830","nonce":31832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":31832},{"type":"v1","addr":"172.21.15.39:6826","nonce":31832}]},"public_addr":"172.21.15.39:6818/31832","cluster_addr":"172.21.15.39:6822/31832","heartbeat_back_addr":"172.21.15.39:6830/31832","heartbeat_front_addr":"172.21.15.39:6826/31832","state":["exists","up"]},{"osd":1,"uuid":"13637dbe-e37f-457f-b68b-8eed1b970617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":31831},{"type":"v1","addr":"172.21.15.39:6809","nonce":31831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":31831},{"type":"v1","addr":"172.21.15.39:6811","nonce":31831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":31831},{"type":"v1","addr":"172.21.15.39:6815","nonce":31831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":31831},{"type":"v1","addr":"172.21.15.39:6813","nonce":31831}]},"public_addr":"172.21.15.39:6809/31831","cluster_addr":"172.21.15.39:6811/31831","heartbeat_back_addr":"172.21.15.39:6815/31831","heartbeat_front_addr":"172.21.15.39:6813/31831","state":["exists","up"]},{"osd":2,"uuid":"29e20ce2-4776-455d-8cf4-d539e738a6a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":31829},{"type":"v1","addr":"172.21.15.39:6801","nonce":31829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":31829},{"type":"v1","addr":"172.21.15.39:6803","nonce":31829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":31829},{"type":"v1","addr":"172.21.15.39:6807","nonce":31829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":31829},{"type":"v1","addr":"172.21.15.39:6805","nonce":31829}]},"public_addr":"172.21.15.39:6801/31829","cluster_addr":"172.21.15.39:6803/31829","heartbeat_back_addr":"172.21.15.39:6807/31829","heartbeat_front_addr":"172.21.15.39:6805/31829","state":["exists","up"]},{"osd":3,"uuid":"a8b34e02-0898-4303-9fd5-62cbba9fec5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6817","nonce":31830},{"type":"v1","addr":"172.21.15.39:6819","nonce":31830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6821","nonce":31830},{"type":"v1","addr":"172.21.15.39:6823","nonce":31830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6829","nonce":31830},{"type":"v1","addr":"172.21.15.39:6831","nonce":31830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6825","nonce":31830},{"type":"v1","addr":"172.21.15.39:6827","nonce":31830}]},"public_addr":"172.21.15.39:6819/31830","cluster_addr":"172.21.15.39:6823/31830","heartbeat_back_addr":"172.21.15.39:6831/31830","heartbeat_front_addr":"172.21.15.39:6827/31830","state":["exists","up"]},{"osd":4,"uuid":"e92b5c67-6908-4fd4-9940-5cb98b5ee847","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":31724},{"type":"v1","addr":"172.21.15.135:6801","nonce":31724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":31724},{"type":"v1","addr":"172.21.15.135:6803","nonce":31724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":31724},{"type":"v1","addr":"172.21.15.135:6807","nonce":31724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":31724},{"type":"v1","addr":"172.21.15.135:6805","nonce":31724}]},"public_addr":"172.21.15.135:6801/31724","cluster_addr":"172.21.15.135:6803/31724","heartbeat_back_addr":"172.21.15.135:6807/31724","heartbeat_front_addr":"172.21.15.135:6805/31724","state":["exists","up"]},{"osd":5,"uuid":"d74d8fef-0211-413b-bff2-f97776fc1faa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":31734},{"type":"v1","addr":"172.21.15.135:6822","nonce":31734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":31734},{"type":"v1","addr":"172.21.15.135:6826","nonce":31734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":31734},{"type":"v1","addr":"172.21.15.135:6831","nonce":31734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":31734},{"type":"v1","addr":"172.21.15.135:6829","nonce":31734}]},"public_addr":"172.21.15.135:6822/31734","cluster_addr":"172.21.15.135:6826/31734","heartbeat_back_addr":"172.21.15.135:6831/31734","heartbeat_front_addr":"172.21.15.135:6829/31734","state":["exists","up"]},{"osd":6,"uuid":"7dc84c6e-76d8-4f22-8b17-51f62765762d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":31725},{"type":"v1","addr":"172.21.15.135:6809","nonce":31725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":31725},{"type":"v1","addr":"172.21.15.135:6811","nonce":31725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":31725},{"type":"v1","addr":"172.21.15.135:6815","nonce":31725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":31725},{"type":"v1","addr":"172.21.15.135:6813","nonce":31725}]},"public_addr":"172.21.15.135:6809/31725","cluster_addr":"172.21.15.135:6811/31725","heartbeat_back_addr":"172.21.15.135:6815/31725","heartbeat_front_addr":"172.21.15.135:6813/31725","state":["exists","up"]},{"osd":7,"uuid":"7adb3187-aed3-48af-868c-7230e9bfc85e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":31737},{"type":"v1","addr":"172.21.15.135:6817","nonce":31737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":31737},{"type":"v1","addr":"172.21.15.135:6819","nonce":31737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":31737},{"type":"v1","addr":"172.21.15.135:6827","nonce":31737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":31737},{"type":"v1","addr":"172.21.15.135:6823","nonce":31737}]},"public_addr":"172.21.15.135:6817/31737","cluster_addr":"172.21.15.135:6819/31737","heartbeat_back_addr":"172.21.15.135:6827/31737","heartbeat_front_addr":"172.21.15.135:6823/31737","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.683465+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.722423+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.712691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.673892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.768469+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.783897+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.686833+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.739675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T20:09:16.449 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T20:09:16.449 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T20:09:16.740 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:09:16.740 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":15,"fsid":"975ceafd-555d-4f71-81e4-1f70364da00f","created":"2024-04-25T20:09:03.377320+0000","modified":"2024-04-25T20:09:15.745353+0000","last_up_change":"2024-04-25T20:09:13.740341+0000","last_in_change":"2024-04-25T20:09:05.529264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-25T20:09:14.202057+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-04-25T20:09:15.645688+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7a7118ba-4aee-404f-97e3-94c50e2996c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":31832},{"type":"v1","addr":"172.21.15.39:6818","nonce":31832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":31832},{"type":"v1","addr":"172.21.15.39:6822","nonce":31832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":31832},{"type":"v1","addr":"172.21.15.39:6830","nonce":31832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":31832},{"type":"v1","addr":"172.21.15.39:6826","nonce":31832}]},"public_addr":"172.21.15.39:6818/31832","cluster_addr":"172.21.15.39:6822/31832","heartbeat_back_addr":"172.21.15.39:6830/31832","heartbeat_front_addr":"172.21.15.39:6826/31832","state":["exists","up"]},{"osd":1,"uuid":"13637dbe-e37f-457f-b68b-8eed1b970617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":31831},{"type":"v1","addr":"172.21.15.39:6809","nonce":31831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":31831},{"type":"v1","addr":"172.21.15.39:6811","nonce":31831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":31831},{"type":"v1","addr":"172.21.15.39:6815","nonce":31831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":31831},{"type":"v1","addr":"172.21.15.39:6813","nonce":31831}]},"public_addr":"172.21.15.39:6809/31831","cluster_addr":"172.21.15.39:6811/31831","heartbeat_back_addr":"172.21.15.39:6815/31831","heartbeat_front_addr":"172.21.15.39:6813/31831","state":["exists","up"]},{"osd":2,"uuid":"29e20ce2-4776-455d-8cf4-d539e738a6a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":31829},{"type":"v1","addr":"172.21.15.39:6801","nonce":31829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":31829},{"type":"v1","addr":"172.21.15.39:6803","nonce":31829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":31829},{"type":"v1","addr":"172.21.15.39:6807","nonce":31829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":31829},{"type":"v1","addr":"172.21.15.39:6805","nonce":31829}]},"public_addr":"172.21.15.39:6801/31829","cluster_addr":"172.21.15.39:6803/31829","heartbeat_back_addr":"172.21.15.39:6807/31829","heartbeat_front_addr":"172.21.15.39:6805/31829","state":["exists","up"]},{"osd":3,"uuid":"a8b34e02-0898-4303-9fd5-62cbba9fec5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6817","nonce":31830},{"type":"v1","addr":"172.21.15.39:6819","nonce":31830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6821","nonce":31830},{"type":"v1","addr":"172.21.15.39:6823","nonce":31830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6829","nonce":31830},{"type":"v1","addr":"172.21.15.39:6831","nonce":31830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6825","nonce":31830},{"type":"v1","addr":"172.21.15.39:6827","nonce":31830}]},"public_addr":"172.21.15.39:6819/31830","cluster_addr":"172.21.15.39:6823/31830","heartbeat_back_addr":"172.21.15.39:6831/31830","heartbeat_front_addr":"172.21.15.39:6827/31830","state":["exists","up"]},{"osd":4,"uuid":"e92b5c67-6908-4fd4-9940-5cb98b5ee847","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":31724},{"type":"v1","addr":"172.21.15.135:6801","nonce":31724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":31724},{"type":"v1","addr":"172.21.15.135:6803","nonce":31724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":31724},{"type":"v1","addr":"172.21.15.135:6807","nonce":31724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":31724},{"type":"v1","addr":"172.21.15.135:6805","nonce":31724}]},"public_addr":"172.21.15.135:6801/31724","cluster_addr":"172.21.15.135:6803/31724","heartbeat_back_addr":"172.21.15.135:6807/31724","heartbeat_front_addr":"172.21.15.135:6805/31724","state":["exists","up"]},{"osd":5,"uuid":"d74d8fef-0211-413b-bff2-f97776fc1faa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":31734},{"type":"v1","addr":"172.21.15.135:6822","nonce":31734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":31734},{"type":"v1","addr":"172.21.15.135:6826","nonce":31734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":31734},{"type":"v1","addr":"172.21.15.135:6831","nonce":31734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":31734},{"type":"v1","addr":"172.21.15.135:6829","nonce":31734}]},"public_addr":"172.21.15.135:6822/31734","cluster_addr":"172.21.15.135:6826/31734","heartbeat_back_addr":"172.21.15.135:6831/31734","heartbeat_front_addr":"172.21.15.135:6829/31734","state":["exists","up"]},{"osd":6,"uuid":"7dc84c6e-76d8-4f22-8b17-51f62765762d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":31725},{"type":"v1","addr":"172.21.15.135:6809","nonce":31725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":31725},{"type":"v1","addr":"172.21.15.135:6811","nonce":31725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":31725},{"type":"v1","addr":"172.21.15.135:6815","nonce":31725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":31725},{"type":"v1","addr":"172.21.15.135:6813","nonce":31725}]},"public_addr":"172.21.15.135:6809/31725","cluster_addr":"172.21.15.135:6811/31725","heartbeat_back_addr":"172.21.15.135:6815/31725","heartbeat_front_addr":"172.21.15.135:6813/31725","state":["exists","up"]},{"osd":7,"uuid":"7adb3187-aed3-48af-868c-7230e9bfc85e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":31737},{"type":"v1","addr":"172.21.15.135:6817","nonce":31737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":31737},{"type":"v1","addr":"172.21.15.135:6819","nonce":31737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":31737},{"type":"v1","addr":"172.21.15.135:6827","nonce":31737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":31737},{"type":"v1","addr":"172.21.15.135:6823","nonce":31737}]},"public_addr":"172.21.15.135:6817/31737","cluster_addr":"172.21.15.135:6819/31737","heartbeat_back_addr":"172.21.15.135:6827/31737","heartbeat_front_addr":"172.21.15.135:6823/31737","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.683465+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.722423+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.712691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.673892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.768469+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.783897+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.686833+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.739675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T20:09:16.753 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T20:09:16.753 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T20:09:16.754 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T20:09:16.754 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T20:09:16.755 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T20:09:16.755 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T20:09:16.755 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T20:09:16.756 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T20:09:17.006 INFO:teuthology.orchestra.run.smithi039.stdout:55834574851 2024-04-25T20:09:17.006 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T20:09:17.007 INFO:teuthology.orchestra.run.smithi039.stdout:55834574851 2024-04-25T20:09:17.008 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T20:09:17.009 INFO:teuthology.orchestra.run.smithi039.stdout:55834574851 2024-04-25T20:09:17.009 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T20:09:17.011 INFO:teuthology.orchestra.run.smithi039.stdout:55834574852 2024-04-25T20:09:17.011 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T20:09:17.012 INFO:teuthology.orchestra.run.smithi039.stdout:55834574851 2024-04-25T20:09:17.013 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T20:09:17.014 INFO:teuthology.orchestra.run.smithi039.stdout:55834574851 2024-04-25T20:09:17.014 INFO:teuthology.orchestra.run.smithi039.stdout:55834574851 2024-04-25T20:09:17.015 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T20:09:17.015 INFO:teuthology.orchestra.run.smithi039.stdout:55834574851 2024-04-25T20:09:17.016 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T20:09:17.016 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T20:09:17.606 INFO:teuthology.orchestra.run.smithi039.stdout:55834574850 2024-04-25T20:09:17.631 INFO:teuthology.orchestra.run.smithi039.stdout:55834574850 2024-04-25T20:09:17.632 INFO:teuthology.orchestra.run.smithi039.stdout:55834574850 2024-04-25T20:09:17.633 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.2 2024-04-25T20:09:17.652 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.6 2024-04-25T20:09:17.653 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.7 2024-04-25T20:09:17.668 INFO:teuthology.orchestra.run.smithi039.stdout:55834574850 2024-04-25T20:09:17.671 INFO:teuthology.orchestra.run.smithi039.stdout:55834574850 2024-04-25T20:09:17.685 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.5 2024-04-25T20:09:17.687 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.0 2024-04-25T20:09:17.692 INFO:teuthology.orchestra.run.smithi039.stdout:55834574851 2024-04-25T20:09:17.705 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.3 2024-04-25T20:09:17.735 INFO:teuthology.orchestra.run.smithi039.stdout:55834574850 2024-04-25T20:09:17.746 INFO:teuthology.orchestra.run.smithi039.stdout:55834574850 2024-04-25T20:09:17.752 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.4 2024-04-25T20:09:17.760 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.1 2024-04-25T20:09:18.633 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T20:09:18.653 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T20:09:18.654 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T20:09:18.685 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T20:09:18.688 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T20:09:18.706 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T20:09:18.753 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T20:09:18.760 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T20:09:19.181 INFO:teuthology.orchestra.run.smithi039.stdout:55834574851 2024-04-25T20:09:19.202 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.2 2024-04-25T20:09:19.202 DEBUG:teuthology.parallel:result is None 2024-04-25T20:09:19.245 INFO:teuthology.orchestra.run.smithi039.stdout:55834574851 2024-04-25T20:09:19.254 INFO:teuthology.orchestra.run.smithi039.stdout:55834574851 2024-04-25T20:09:19.264 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.7 2024-04-25T20:09:19.264 DEBUG:teuthology.parallel:result is None 2024-04-25T20:09:19.265 INFO:teuthology.orchestra.run.smithi039.stdout:55834574851 2024-04-25T20:09:19.276 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.6 2024-04-25T20:09:19.276 DEBUG:teuthology.parallel:result is None 2024-04-25T20:09:19.282 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.0 2024-04-25T20:09:19.283 DEBUG:teuthology.parallel:result is None 2024-04-25T20:09:19.299 INFO:teuthology.orchestra.run.smithi039.stdout:55834574851 2024-04-25T20:09:19.310 INFO:teuthology.orchestra.run.smithi039.stdout:55834574852 2024-04-25T20:09:19.313 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.5 2024-04-25T20:09:19.313 DEBUG:teuthology.parallel:result is None 2024-04-25T20:09:19.320 INFO:teuthology.orchestra.run.smithi039.stdout:55834574851 2024-04-25T20:09:19.330 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.3 2024-04-25T20:09:19.330 DEBUG:teuthology.parallel:result is None 2024-04-25T20:09:19.334 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.1 2024-04-25T20:09:19.334 DEBUG:teuthology.parallel:result is None 2024-04-25T20:09:19.351 INFO:teuthology.orchestra.run.smithi039.stdout:55834574851 2024-04-25T20:09:19.364 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.4 2024-04-25T20:09:19.364 DEBUG:teuthology.parallel:result is None 2024-04-25T20:09:19.364 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T20:09:19.364 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T20:09:19.699 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:09:19.700 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-04-25T20:09:19.712 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-04-25T20:09:19.603061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233700,"kb_used_data":1140,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754741020,"statfs":{"total":773094113280,"available":772854804480,"internally_reserved":0,"allocated":1167360,"data_stored":339180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.759430+0000","last_change":"2024-04-25T20:09:15.769542+0000","last_active":"2024-04-25T20:09:16.759430+0000","last_peered":"2024-04-25T20:09:16.759430+0000","last_clean":"2024-04-25T20:09:16.759430+0000","last_became_active":"2024-04-25T20:09:15.769271+0000","last_became_peered":"2024-04-25T20:09:15.769271+0000","last_unstale":"2024-04-25T20:09:16.759430+0000","last_undegraded":"2024-04-25T20:09:16.759430+0000","last_fullsized":"2024-04-25T20:09:16.759430+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:31.359804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.759433+0000","last_change":"2024-04-25T20:09:16.141209+0000","last_active":"2024-04-25T20:09:16.759433+0000","last_peered":"2024-04-25T20:09:16.759433+0000","last_clean":"2024-04-25T20:09:16.759433+0000","last_became_active":"2024-04-25T20:09:16.140616+0000","last_became_peered":"2024-04-25T20:09:16.140616+0000","last_unstale":"2024-04-25T20:09:16.759433+0000","last_undegraded":"2024-04-25T20:09:16.759433+0000","last_fullsized":"2024-04-25T20:09:16.759433+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:31.562175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-25T20:09:15.767070+0000","last_change":"2024-04-25T20:09:15.762535+0000","last_active":"2024-04-25T20:09:15.767070+0000","last_peered":"2024-04-25T20:09:15.767070+0000","last_clean":"2024-04-25T20:09:15.767070+0000","last_became_active":"2024-04-25T20:09:15.762247+0000","last_became_peered":"2024-04-25T20:09:15.762247+0000","last_unstale":"2024-04-25T20:09:15.767070+0000","last_undegraded":"2024-04-25T20:09:15.767070+0000","last_fullsized":"2024-04-25T20:09:15.767070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:06:25.576044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-25T20:09:15.775837+0000","last_change":"2024-04-25T20:09:15.775252+0000","last_active":"2024-04-25T20:09:15.775837+0000","last_peered":"2024-04-25T20:09:15.775837+0000","last_clean":"2024-04-25T20:09:15.775837+0000","last_became_active":"2024-04-25T20:09:15.775027+0000","last_became_peered":"2024-04-25T20:09:15.775027+0000","last_unstale":"2024-04-25T20:09:15.775837+0000","last_undegraded":"2024-04-25T20:09:15.775837+0000","last_fullsized":"2024-04-25T20:09:15.775837+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:58.259077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":3,"reported_epoch":16,"state":"creating+activating","last_fresh":"2024-04-25T20:09:16.760520+0000","last_change":"2024-04-25T20:09:16.760520+0000","last_active":"2024-04-25T20:09:15.745353+0000","last_peered":"2024-04-25T20:09:15.745353+0000","last_clean":"2024-04-25T20:09:15.745353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T20:09:16.760520+0000","last_undegraded":"2024-04-25T20:09:16.760520+0000","last_fullsized":"2024-04-25T20:09:16.760520+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_clean_scrub_stamp":"2024-04-25T20:09:15.745353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:48.383841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":["7","1"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-25T20:09:15.773363+0000","last_change":"2024-04-25T20:09:15.772839+0000","last_active":"2024-04-25T20:09:15.773363+0000","last_peered":"2024-04-25T20:09:15.773363+0000","last_clean":"2024-04-25T20:09:15.773363+0000","last_became_active":"2024-04-25T20:09:15.772551+0000","last_became_peered":"2024-04-25T20:09:15.772551+0000","last_unstale":"2024-04-25T20:09:15.773363+0000","last_undegraded":"2024-04-25T20:09:15.773363+0000","last_fullsized":"2024-04-25T20:09:15.773363+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:55.101025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.760389+0000","last_change":"2024-04-25T20:09:16.141037+0000","last_active":"2024-04-25T20:09:16.760389+0000","last_peered":"2024-04-25T20:09:16.760389+0000","last_clean":"2024-04-25T20:09:16.760389+0000","last_became_active":"2024-04-25T20:09:16.140704+0000","last_became_peered":"2024-04-25T20:09:16.140704+0000","last_unstale":"2024-04-25T20:09:16.760389+0000","last_undegraded":"2024-04-25T20:09:16.760389+0000","last_fullsized":"2024-04-25T20:09:16.760389+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:22.537137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.144058+0000","last_change":"2024-04-25T20:09:16.140930+0000","last_active":"2024-04-25T20:09:16.144058+0000","last_peered":"2024-04-25T20:09:16.144058+0000","last_clean":"2024-04-25T20:09:16.144058+0000","last_became_active":"2024-04-25T20:09:16.140617+0000","last_became_peered":"2024-04-25T20:09:16.140617+0000","last_unstale":"2024-04-25T20:09:16.144058+0000","last_undegraded":"2024-04-25T20:09:16.144058+0000","last_fullsized":"2024-04-25T20:09:16.144058+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:41:45.137982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.760372+0000","last_change":"2024-04-25T20:09:16.141064+0000","last_active":"2024-04-25T20:09:16.760372+0000","last_peered":"2024-04-25T20:09:16.760372+0000","last_clean":"2024-04-25T20:09:16.760372+0000","last_became_active":"2024-04-25T20:09:16.140731+0000","last_became_peered":"2024-04-25T20:09:16.140731+0000","last_unstale":"2024-04-25T20:09:16.760372+0000","last_undegraded":"2024-04-25T20:09:16.760372+0000","last_fullsized":"2024-04-25T20:09:16.760372+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:18.458296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":47109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":47109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":143360,"data_stored":40983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":40983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31256,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340584,"statfs":{"total":96636764160,"available":96604758016,"internally_reserved":0,"allocated":143360,"data_stored":40983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31256,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340584,"statfs":{"total":96636764160,"available":96604758016,"internally_reserved":0,"allocated":143360,"data_stored":40983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31256,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340584,"statfs":{"total":96636764160,"available":96604758016,"internally_reserved":0,"allocated":143360,"data_stored":40983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31252,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340588,"statfs":{"total":96636764160,"available":96604762112,"internally_reserved":0,"allocated":139264,"data_stored":40047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:09:19.713 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T20:09:20.003 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:09:20.003 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-04-25T20:09:20.015 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-04-25T20:09:19.603061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233700,"kb_used_data":1140,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754741020,"statfs":{"total":773094113280,"available":772854804480,"internally_reserved":0,"allocated":1167360,"data_stored":339180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.759430+0000","last_change":"2024-04-25T20:09:15.769542+0000","last_active":"2024-04-25T20:09:16.759430+0000","last_peered":"2024-04-25T20:09:16.759430+0000","last_clean":"2024-04-25T20:09:16.759430+0000","last_became_active":"2024-04-25T20:09:15.769271+0000","last_became_peered":"2024-04-25T20:09:15.769271+0000","last_unstale":"2024-04-25T20:09:16.759430+0000","last_undegraded":"2024-04-25T20:09:16.759430+0000","last_fullsized":"2024-04-25T20:09:16.759430+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:31.359804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.759433+0000","last_change":"2024-04-25T20:09:16.141209+0000","last_active":"2024-04-25T20:09:16.759433+0000","last_peered":"2024-04-25T20:09:16.759433+0000","last_clean":"2024-04-25T20:09:16.759433+0000","last_became_active":"2024-04-25T20:09:16.140616+0000","last_became_peered":"2024-04-25T20:09:16.140616+0000","last_unstale":"2024-04-25T20:09:16.759433+0000","last_undegraded":"2024-04-25T20:09:16.759433+0000","last_fullsized":"2024-04-25T20:09:16.759433+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:31.562175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-25T20:09:15.767070+0000","last_change":"2024-04-25T20:09:15.762535+0000","last_active":"2024-04-25T20:09:15.767070+0000","last_peered":"2024-04-25T20:09:15.767070+0000","last_clean":"2024-04-25T20:09:15.767070+0000","last_became_active":"2024-04-25T20:09:15.762247+0000","last_became_peered":"2024-04-25T20:09:15.762247+0000","last_unstale":"2024-04-25T20:09:15.767070+0000","last_undegraded":"2024-04-25T20:09:15.767070+0000","last_fullsized":"2024-04-25T20:09:15.767070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:06:25.576044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-25T20:09:15.775837+0000","last_change":"2024-04-25T20:09:15.775252+0000","last_active":"2024-04-25T20:09:15.775837+0000","last_peered":"2024-04-25T20:09:15.775837+0000","last_clean":"2024-04-25T20:09:15.775837+0000","last_became_active":"2024-04-25T20:09:15.775027+0000","last_became_peered":"2024-04-25T20:09:15.775027+0000","last_unstale":"2024-04-25T20:09:15.775837+0000","last_undegraded":"2024-04-25T20:09:15.775837+0000","last_fullsized":"2024-04-25T20:09:15.775837+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:58.259077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":3,"reported_epoch":16,"state":"creating+activating","last_fresh":"2024-04-25T20:09:16.760520+0000","last_change":"2024-04-25T20:09:16.760520+0000","last_active":"2024-04-25T20:09:15.745353+0000","last_peered":"2024-04-25T20:09:15.745353+0000","last_clean":"2024-04-25T20:09:15.745353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T20:09:16.760520+0000","last_undegraded":"2024-04-25T20:09:16.760520+0000","last_fullsized":"2024-04-25T20:09:16.760520+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_clean_scrub_stamp":"2024-04-25T20:09:15.745353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:48.383841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":["7","1"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-25T20:09:15.773363+0000","last_change":"2024-04-25T20:09:15.772839+0000","last_active":"2024-04-25T20:09:15.773363+0000","last_peered":"2024-04-25T20:09:15.773363+0000","last_clean":"2024-04-25T20:09:15.773363+0000","last_became_active":"2024-04-25T20:09:15.772551+0000","last_became_peered":"2024-04-25T20:09:15.772551+0000","last_unstale":"2024-04-25T20:09:15.773363+0000","last_undegraded":"2024-04-25T20:09:15.773363+0000","last_fullsized":"2024-04-25T20:09:15.773363+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:55.101025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.760389+0000","last_change":"2024-04-25T20:09:16.141037+0000","last_active":"2024-04-25T20:09:16.760389+0000","last_peered":"2024-04-25T20:09:16.760389+0000","last_clean":"2024-04-25T20:09:16.760389+0000","last_became_active":"2024-04-25T20:09:16.140704+0000","last_became_peered":"2024-04-25T20:09:16.140704+0000","last_unstale":"2024-04-25T20:09:16.760389+0000","last_undegraded":"2024-04-25T20:09:16.760389+0000","last_fullsized":"2024-04-25T20:09:16.760389+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:22.537137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.144058+0000","last_change":"2024-04-25T20:09:16.140930+0000","last_active":"2024-04-25T20:09:16.144058+0000","last_peered":"2024-04-25T20:09:16.144058+0000","last_clean":"2024-04-25T20:09:16.144058+0000","last_became_active":"2024-04-25T20:09:16.140617+0000","last_became_peered":"2024-04-25T20:09:16.140617+0000","last_unstale":"2024-04-25T20:09:16.144058+0000","last_undegraded":"2024-04-25T20:09:16.144058+0000","last_fullsized":"2024-04-25T20:09:16.144058+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:41:45.137982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.760372+0000","last_change":"2024-04-25T20:09:16.141064+0000","last_active":"2024-04-25T20:09:16.760372+0000","last_peered":"2024-04-25T20:09:16.760372+0000","last_clean":"2024-04-25T20:09:16.760372+0000","last_became_active":"2024-04-25T20:09:16.140731+0000","last_became_peered":"2024-04-25T20:09:16.140731+0000","last_unstale":"2024-04-25T20:09:16.760372+0000","last_undegraded":"2024-04-25T20:09:16.760372+0000","last_fullsized":"2024-04-25T20:09:16.760372+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:18.458296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":47109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":47109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":143360,"data_stored":40983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":40983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31256,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340584,"statfs":{"total":96636764160,"available":96604758016,"internally_reserved":0,"allocated":143360,"data_stored":40983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31256,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340584,"statfs":{"total":96636764160,"available":96604758016,"internally_reserved":0,"allocated":143360,"data_stored":40983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31256,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340584,"statfs":{"total":96636764160,"available":96604758016,"internally_reserved":0,"allocated":143360,"data_stored":40983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31252,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340588,"statfs":{"total":96636764160,"available":96604762112,"internally_reserved":0,"allocated":139264,"data_stored":40047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:09:20.016 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T20:09:20.307 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:09:20.307 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-04-25T20:09:20.320 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-04-25T20:09:19.603061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233700,"kb_used_data":1140,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754741020,"statfs":{"total":773094113280,"available":772854804480,"internally_reserved":0,"allocated":1167360,"data_stored":339180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.759430+0000","last_change":"2024-04-25T20:09:15.769542+0000","last_active":"2024-04-25T20:09:16.759430+0000","last_peered":"2024-04-25T20:09:16.759430+0000","last_clean":"2024-04-25T20:09:16.759430+0000","last_became_active":"2024-04-25T20:09:15.769271+0000","last_became_peered":"2024-04-25T20:09:15.769271+0000","last_unstale":"2024-04-25T20:09:16.759430+0000","last_undegraded":"2024-04-25T20:09:16.759430+0000","last_fullsized":"2024-04-25T20:09:16.759430+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:31.359804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.759433+0000","last_change":"2024-04-25T20:09:16.141209+0000","last_active":"2024-04-25T20:09:16.759433+0000","last_peered":"2024-04-25T20:09:16.759433+0000","last_clean":"2024-04-25T20:09:16.759433+0000","last_became_active":"2024-04-25T20:09:16.140616+0000","last_became_peered":"2024-04-25T20:09:16.140616+0000","last_unstale":"2024-04-25T20:09:16.759433+0000","last_undegraded":"2024-04-25T20:09:16.759433+0000","last_fullsized":"2024-04-25T20:09:16.759433+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:31.562175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-25T20:09:15.767070+0000","last_change":"2024-04-25T20:09:15.762535+0000","last_active":"2024-04-25T20:09:15.767070+0000","last_peered":"2024-04-25T20:09:15.767070+0000","last_clean":"2024-04-25T20:09:15.767070+0000","last_became_active":"2024-04-25T20:09:15.762247+0000","last_became_peered":"2024-04-25T20:09:15.762247+0000","last_unstale":"2024-04-25T20:09:15.767070+0000","last_undegraded":"2024-04-25T20:09:15.767070+0000","last_fullsized":"2024-04-25T20:09:15.767070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:06:25.576044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-25T20:09:15.775837+0000","last_change":"2024-04-25T20:09:15.775252+0000","last_active":"2024-04-25T20:09:15.775837+0000","last_peered":"2024-04-25T20:09:15.775837+0000","last_clean":"2024-04-25T20:09:15.775837+0000","last_became_active":"2024-04-25T20:09:15.775027+0000","last_became_peered":"2024-04-25T20:09:15.775027+0000","last_unstale":"2024-04-25T20:09:15.775837+0000","last_undegraded":"2024-04-25T20:09:15.775837+0000","last_fullsized":"2024-04-25T20:09:15.775837+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:58.259077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":3,"reported_epoch":16,"state":"creating+activating","last_fresh":"2024-04-25T20:09:16.760520+0000","last_change":"2024-04-25T20:09:16.760520+0000","last_active":"2024-04-25T20:09:15.745353+0000","last_peered":"2024-04-25T20:09:15.745353+0000","last_clean":"2024-04-25T20:09:15.745353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T20:09:16.760520+0000","last_undegraded":"2024-04-25T20:09:16.760520+0000","last_fullsized":"2024-04-25T20:09:16.760520+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_clean_scrub_stamp":"2024-04-25T20:09:15.745353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:48.383841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":["7","1"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-25T20:09:15.773363+0000","last_change":"2024-04-25T20:09:15.772839+0000","last_active":"2024-04-25T20:09:15.773363+0000","last_peered":"2024-04-25T20:09:15.773363+0000","last_clean":"2024-04-25T20:09:15.773363+0000","last_became_active":"2024-04-25T20:09:15.772551+0000","last_became_peered":"2024-04-25T20:09:15.772551+0000","last_unstale":"2024-04-25T20:09:15.773363+0000","last_undegraded":"2024-04-25T20:09:15.773363+0000","last_fullsized":"2024-04-25T20:09:15.773363+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:55.101025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.760389+0000","last_change":"2024-04-25T20:09:16.141037+0000","last_active":"2024-04-25T20:09:16.760389+0000","last_peered":"2024-04-25T20:09:16.760389+0000","last_clean":"2024-04-25T20:09:16.760389+0000","last_became_active":"2024-04-25T20:09:16.140704+0000","last_became_peered":"2024-04-25T20:09:16.140704+0000","last_unstale":"2024-04-25T20:09:16.760389+0000","last_undegraded":"2024-04-25T20:09:16.760389+0000","last_fullsized":"2024-04-25T20:09:16.760389+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:22.537137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.144058+0000","last_change":"2024-04-25T20:09:16.140930+0000","last_active":"2024-04-25T20:09:16.144058+0000","last_peered":"2024-04-25T20:09:16.144058+0000","last_clean":"2024-04-25T20:09:16.144058+0000","last_became_active":"2024-04-25T20:09:16.140617+0000","last_became_peered":"2024-04-25T20:09:16.140617+0000","last_unstale":"2024-04-25T20:09:16.144058+0000","last_undegraded":"2024-04-25T20:09:16.144058+0000","last_fullsized":"2024-04-25T20:09:16.144058+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:41:45.137982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.760372+0000","last_change":"2024-04-25T20:09:16.141064+0000","last_active":"2024-04-25T20:09:16.760372+0000","last_peered":"2024-04-25T20:09:16.760372+0000","last_clean":"2024-04-25T20:09:16.760372+0000","last_became_active":"2024-04-25T20:09:16.140731+0000","last_became_peered":"2024-04-25T20:09:16.140731+0000","last_unstale":"2024-04-25T20:09:16.760372+0000","last_undegraded":"2024-04-25T20:09:16.760372+0000","last_fullsized":"2024-04-25T20:09:16.760372+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:18.458296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":47109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":47109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":143360,"data_stored":40983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":40983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31256,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340584,"statfs":{"total":96636764160,"available":96604758016,"internally_reserved":0,"allocated":143360,"data_stored":40983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31256,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340584,"statfs":{"total":96636764160,"available":96604758016,"internally_reserved":0,"allocated":143360,"data_stored":40983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31256,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340584,"statfs":{"total":96636764160,"available":96604758016,"internally_reserved":0,"allocated":143360,"data_stored":40983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31252,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340588,"statfs":{"total":96636764160,"available":96604762112,"internally_reserved":0,"allocated":139264,"data_stored":40047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:09:20.321 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2024-04-25T20:09:20.321 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 16, 'state': 'creating+activating', 'last_fresh': '2024-04-25T20:09:16.760520+0000', 'last_change': '2024-04-25T20:09:16.760520+0000', 'last_active': '2024-04-25T20:09:15.745353+0000', 'last_peered': '2024-04-25T20:09:15.745353+0000', 'last_clean': '2024-04-25T20:09:15.745353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T20:09:16.760520+0000', 'last_undegraded': '2024-04-25T20:09:16.760520+0000', 'last_fullsized': '2024-04-25T20:09:16.760520+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T20:09:15.745353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T20:09:15.745353+0000', 'last_clean_scrub_stamp': '2024-04-25T20:09:15.745353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-04-26T20:43:48.383841+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': ['7', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T20:09:20.321 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T20:09:20.683 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:09:20.683 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"975ceafd-555d-4f71-81e4-1f70364da00f","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":17,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":17,"num_osds":8,"num_up_osds":8,"osd_up_since":1714075753,"num_in_osds":8,"osd_in_since":1714075745,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":8},{"state_name":"creating+activating","count":1}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":239308800,"bytes_avail":772854804480,"bytes_total":773094113280,"inactive_pgs_ratio":0.1111111119389534},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-04-25T20:09:03.378678+0000","services":{}},"progress_events":{}} 2024-04-25T20:09:20.696 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-04-25T20:09:20.696 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T20:09:20.987 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:09:20.988 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-04-25T20:09:21.000 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-04-25T20:09:19.603061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233700,"kb_used_data":1140,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754741020,"statfs":{"total":773094113280,"available":772854804480,"internally_reserved":0,"allocated":1167360,"data_stored":339180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.759430+0000","last_change":"2024-04-25T20:09:15.769542+0000","last_active":"2024-04-25T20:09:16.759430+0000","last_peered":"2024-04-25T20:09:16.759430+0000","last_clean":"2024-04-25T20:09:16.759430+0000","last_became_active":"2024-04-25T20:09:15.769271+0000","last_became_peered":"2024-04-25T20:09:15.769271+0000","last_unstale":"2024-04-25T20:09:16.759430+0000","last_undegraded":"2024-04-25T20:09:16.759430+0000","last_fullsized":"2024-04-25T20:09:16.759430+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:31.359804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.759433+0000","last_change":"2024-04-25T20:09:16.141209+0000","last_active":"2024-04-25T20:09:16.759433+0000","last_peered":"2024-04-25T20:09:16.759433+0000","last_clean":"2024-04-25T20:09:16.759433+0000","last_became_active":"2024-04-25T20:09:16.140616+0000","last_became_peered":"2024-04-25T20:09:16.140616+0000","last_unstale":"2024-04-25T20:09:16.759433+0000","last_undegraded":"2024-04-25T20:09:16.759433+0000","last_fullsized":"2024-04-25T20:09:16.759433+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:31.562175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-25T20:09:15.767070+0000","last_change":"2024-04-25T20:09:15.762535+0000","last_active":"2024-04-25T20:09:15.767070+0000","last_peered":"2024-04-25T20:09:15.767070+0000","last_clean":"2024-04-25T20:09:15.767070+0000","last_became_active":"2024-04-25T20:09:15.762247+0000","last_became_peered":"2024-04-25T20:09:15.762247+0000","last_unstale":"2024-04-25T20:09:15.767070+0000","last_undegraded":"2024-04-25T20:09:15.767070+0000","last_fullsized":"2024-04-25T20:09:15.767070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:06:25.576044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-25T20:09:15.775837+0000","last_change":"2024-04-25T20:09:15.775252+0000","last_active":"2024-04-25T20:09:15.775837+0000","last_peered":"2024-04-25T20:09:15.775837+0000","last_clean":"2024-04-25T20:09:15.775837+0000","last_became_active":"2024-04-25T20:09:15.775027+0000","last_became_peered":"2024-04-25T20:09:15.775027+0000","last_unstale":"2024-04-25T20:09:15.775837+0000","last_undegraded":"2024-04-25T20:09:15.775837+0000","last_fullsized":"2024-04-25T20:09:15.775837+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:58.259077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":3,"reported_epoch":16,"state":"creating+activating","last_fresh":"2024-04-25T20:09:16.760520+0000","last_change":"2024-04-25T20:09:16.760520+0000","last_active":"2024-04-25T20:09:15.745353+0000","last_peered":"2024-04-25T20:09:15.745353+0000","last_clean":"2024-04-25T20:09:15.745353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T20:09:16.760520+0000","last_undegraded":"2024-04-25T20:09:16.760520+0000","last_fullsized":"2024-04-25T20:09:16.760520+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_clean_scrub_stamp":"2024-04-25T20:09:15.745353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:48.383841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":["7","1"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-25T20:09:15.773363+0000","last_change":"2024-04-25T20:09:15.772839+0000","last_active":"2024-04-25T20:09:15.773363+0000","last_peered":"2024-04-25T20:09:15.773363+0000","last_clean":"2024-04-25T20:09:15.773363+0000","last_became_active":"2024-04-25T20:09:15.772551+0000","last_became_peered":"2024-04-25T20:09:15.772551+0000","last_unstale":"2024-04-25T20:09:15.773363+0000","last_undegraded":"2024-04-25T20:09:15.773363+0000","last_fullsized":"2024-04-25T20:09:15.773363+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:55.101025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.760389+0000","last_change":"2024-04-25T20:09:16.141037+0000","last_active":"2024-04-25T20:09:16.760389+0000","last_peered":"2024-04-25T20:09:16.760389+0000","last_clean":"2024-04-25T20:09:16.760389+0000","last_became_active":"2024-04-25T20:09:16.140704+0000","last_became_peered":"2024-04-25T20:09:16.140704+0000","last_unstale":"2024-04-25T20:09:16.760389+0000","last_undegraded":"2024-04-25T20:09:16.760389+0000","last_fullsized":"2024-04-25T20:09:16.760389+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:22.537137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.144058+0000","last_change":"2024-04-25T20:09:16.140930+0000","last_active":"2024-04-25T20:09:16.144058+0000","last_peered":"2024-04-25T20:09:16.144058+0000","last_clean":"2024-04-25T20:09:16.144058+0000","last_became_active":"2024-04-25T20:09:16.140617+0000","last_became_peered":"2024-04-25T20:09:16.140617+0000","last_unstale":"2024-04-25T20:09:16.144058+0000","last_undegraded":"2024-04-25T20:09:16.144058+0000","last_fullsized":"2024-04-25T20:09:16.144058+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:41:45.137982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.760372+0000","last_change":"2024-04-25T20:09:16.141064+0000","last_active":"2024-04-25T20:09:16.760372+0000","last_peered":"2024-04-25T20:09:16.760372+0000","last_clean":"2024-04-25T20:09:16.760372+0000","last_became_active":"2024-04-25T20:09:16.140731+0000","last_became_peered":"2024-04-25T20:09:16.140731+0000","last_unstale":"2024-04-25T20:09:16.760372+0000","last_undegraded":"2024-04-25T20:09:16.760372+0000","last_fullsized":"2024-04-25T20:09:16.760372+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:18.458296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":47109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":47109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":143360,"data_stored":40983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":40983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31256,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340584,"statfs":{"total":96636764160,"available":96604758016,"internally_reserved":0,"allocated":143360,"data_stored":40983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31256,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340584,"statfs":{"total":96636764160,"available":96604758016,"internally_reserved":0,"allocated":143360,"data_stored":40983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31256,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340584,"statfs":{"total":96636764160,"available":96604758016,"internally_reserved":0,"allocated":143360,"data_stored":40983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31252,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340588,"statfs":{"total":96636764160,"available":96604762112,"internally_reserved":0,"allocated":139264,"data_stored":40047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:09:24.002 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T20:09:24.307 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:09:24.308 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-04-25T20:09:24.319 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-04-25T20:09:23.604059+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234064,"kb_used_data":1420,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754740656,"statfs":{"total":773094113280,"available":772854431744,"internally_reserved":0,"allocated":1454080,"data_stored":1602912,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000574"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.983771+0000","last_change":"2024-04-25T20:09:15.769542+0000","last_active":"2024-04-25T20:09:16.983771+0000","last_peered":"2024-04-25T20:09:16.983771+0000","last_clean":"2024-04-25T20:09:16.983771+0000","last_became_active":"2024-04-25T20:09:15.769271+0000","last_became_peered":"2024-04-25T20:09:15.769271+0000","last_unstale":"2024-04-25T20:09:16.983771+0000","last_undegraded":"2024-04-25T20:09:16.983771+0000","last_fullsized":"2024-04-25T20:09:16.983771+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:31.359804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.983757+0000","last_change":"2024-04-25T20:09:16.141209+0000","last_active":"2024-04-25T20:09:16.983757+0000","last_peered":"2024-04-25T20:09:16.983757+0000","last_clean":"2024-04-25T20:09:16.983757+0000","last_became_active":"2024-04-25T20:09:16.140616+0000","last_became_peered":"2024-04-25T20:09:16.140616+0000","last_unstale":"2024-04-25T20:09:16.983757+0000","last_undegraded":"2024-04-25T20:09:16.983757+0000","last_fullsized":"2024-04-25T20:09:16.983757+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:31.562175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.984751+0000","last_change":"2024-04-25T20:09:15.762535+0000","last_active":"2024-04-25T20:09:16.984751+0000","last_peered":"2024-04-25T20:09:16.984751+0000","last_clean":"2024-04-25T20:09:16.984751+0000","last_became_active":"2024-04-25T20:09:15.762247+0000","last_became_peered":"2024-04-25T20:09:15.762247+0000","last_unstale":"2024-04-25T20:09:16.984751+0000","last_undegraded":"2024-04-25T20:09:16.984751+0000","last_fullsized":"2024-04-25T20:09:16.984751+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:06:25.576044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.985459+0000","last_change":"2024-04-25T20:09:15.775252+0000","last_active":"2024-04-25T20:09:16.985459+0000","last_peered":"2024-04-25T20:09:16.985459+0000","last_clean":"2024-04-25T20:09:16.985459+0000","last_became_active":"2024-04-25T20:09:15.775027+0000","last_became_peered":"2024-04-25T20:09:15.775027+0000","last_unstale":"2024-04-25T20:09:16.985459+0000","last_undegraded":"2024-04-25T20:09:16.985459+0000","last_fullsized":"2024-04-25T20:09:16.985459+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:58.259077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-25T20:09:18.631686+0000","last_change":"2024-04-25T20:09:17.121148+0000","last_active":"2024-04-25T20:09:18.631686+0000","last_peered":"2024-04-25T20:09:18.631686+0000","last_clean":"2024-04-25T20:09:18.631686+0000","last_became_active":"2024-04-25T20:09:17.120312+0000","last_became_peered":"2024-04-25T20:09:17.120312+0000","last_unstale":"2024-04-25T20:09:18.631686+0000","last_undegraded":"2024-04-25T20:09:18.631686+0000","last_fullsized":"2024-04-25T20:09:18.631686+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_clean_scrub_stamp":"2024-04-25T20:09:15.745353+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:48.383841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:17.118607+0000","last_change":"2024-04-25T20:09:15.772839+0000","last_active":"2024-04-25T20:09:17.118607+0000","last_peered":"2024-04-25T20:09:17.118607+0000","last_clean":"2024-04-25T20:09:17.118607+0000","last_became_active":"2024-04-25T20:09:15.772551+0000","last_became_peered":"2024-04-25T20:09:15.772551+0000","last_unstale":"2024-04-25T20:09:17.118607+0000","last_undegraded":"2024-04-25T20:09:17.118607+0000","last_fullsized":"2024-04-25T20:09:17.118607+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:55.101025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.981712+0000","last_change":"2024-04-25T20:09:16.141037+0000","last_active":"2024-04-25T20:09:16.981712+0000","last_peered":"2024-04-25T20:09:16.981712+0000","last_clean":"2024-04-25T20:09:16.981712+0000","last_became_active":"2024-04-25T20:09:16.140704+0000","last_became_peered":"2024-04-25T20:09:16.140704+0000","last_unstale":"2024-04-25T20:09:16.981712+0000","last_undegraded":"2024-04-25T20:09:16.981712+0000","last_fullsized":"2024-04-25T20:09:16.981712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:22.537137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.985574+0000","last_change":"2024-04-25T20:09:16.140930+0000","last_active":"2024-04-25T20:09:16.985574+0000","last_peered":"2024-04-25T20:09:16.985574+0000","last_clean":"2024-04-25T20:09:16.985574+0000","last_became_active":"2024-04-25T20:09:16.140617+0000","last_became_peered":"2024-04-25T20:09:16.140617+0000","last_unstale":"2024-04-25T20:09:16.985574+0000","last_undegraded":"2024-04-25T20:09:16.985574+0000","last_fullsized":"2024-04-25T20:09:16.985574+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:41:45.137982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.981700+0000","last_change":"2024-04-25T20:09:16.141064+0000","last_active":"2024-04-25T20:09:16.981700+0000","last_peered":"2024-04-25T20:09:16.981700+0000","last_clean":"2024-04-25T20:09:16.981700+0000","last_became_active":"2024-04-25T20:09:16.140731+0000","last_became_peered":"2024-04-25T20:09:16.140731+0000","last_unstale":"2024-04-25T20:09:16.981700+0000","last_undegraded":"2024-04-25T20:09:16.981700+0000","last_fullsized":"2024-04-25T20:09:16.981700+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:18.458296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":225280,"data_stored":643257,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":52889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":52889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":167936,"data_stored":52889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31284,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340556,"statfs":{"total":96636764160,"available":96604729344,"internally_reserved":0,"allocated":167936,"data_stored":52889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":167936,"data_stored":52889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":225280,"data_stored":643257,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":163840,"data_stored":51953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:09:24.320 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T20:09:24.320 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T20:09:24.320 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T20:09:24.320 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T20:09:24.653 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:09:24.654 INFO:teuthology.orchestra.run.smithi039.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T20:09:24.666 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T20:09:24.667 INFO:teuthology.run_tasks:Running task thrashosds... 2024-04-25T20:09:24.702 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-04-25T20:09:24.702 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '646b8c7d293d8ec31a40c69f7b3237b19c1be24d'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '646b8c7d293d8ec31a40c69f7b3237b19c1be24d'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': '19b26365dac54ee709df8c5dc2a7b7e89a50aba5'}} 2024-04-25T20:09:24.703 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-04-25T20:09:24.703 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-04-25T20:09:25.017 INFO:tasks.thrashosds:Beginning thrashosds... 2024-04-25T20:09:25.017 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T20:09:25.017 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T20:09:25.308 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:09:25.309 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-04-25T20:09:25.320 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-04-25T20:09:23.604059+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234064,"kb_used_data":1420,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754740656,"statfs":{"total":773094113280,"available":772854431744,"internally_reserved":0,"allocated":1454080,"data_stored":1602912,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000574"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.983771+0000","last_change":"2024-04-25T20:09:15.769542+0000","last_active":"2024-04-25T20:09:16.983771+0000","last_peered":"2024-04-25T20:09:16.983771+0000","last_clean":"2024-04-25T20:09:16.983771+0000","last_became_active":"2024-04-25T20:09:15.769271+0000","last_became_peered":"2024-04-25T20:09:15.769271+0000","last_unstale":"2024-04-25T20:09:16.983771+0000","last_undegraded":"2024-04-25T20:09:16.983771+0000","last_fullsized":"2024-04-25T20:09:16.983771+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:31.359804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.983757+0000","last_change":"2024-04-25T20:09:16.141209+0000","last_active":"2024-04-25T20:09:16.983757+0000","last_peered":"2024-04-25T20:09:16.983757+0000","last_clean":"2024-04-25T20:09:16.983757+0000","last_became_active":"2024-04-25T20:09:16.140616+0000","last_became_peered":"2024-04-25T20:09:16.140616+0000","last_unstale":"2024-04-25T20:09:16.983757+0000","last_undegraded":"2024-04-25T20:09:16.983757+0000","last_fullsized":"2024-04-25T20:09:16.983757+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:31.562175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.984751+0000","last_change":"2024-04-25T20:09:15.762535+0000","last_active":"2024-04-25T20:09:16.984751+0000","last_peered":"2024-04-25T20:09:16.984751+0000","last_clean":"2024-04-25T20:09:16.984751+0000","last_became_active":"2024-04-25T20:09:15.762247+0000","last_became_peered":"2024-04-25T20:09:15.762247+0000","last_unstale":"2024-04-25T20:09:16.984751+0000","last_undegraded":"2024-04-25T20:09:16.984751+0000","last_fullsized":"2024-04-25T20:09:16.984751+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:06:25.576044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.985459+0000","last_change":"2024-04-25T20:09:15.775252+0000","last_active":"2024-04-25T20:09:16.985459+0000","last_peered":"2024-04-25T20:09:16.985459+0000","last_clean":"2024-04-25T20:09:16.985459+0000","last_became_active":"2024-04-25T20:09:15.775027+0000","last_became_peered":"2024-04-25T20:09:15.775027+0000","last_unstale":"2024-04-25T20:09:16.985459+0000","last_undegraded":"2024-04-25T20:09:16.985459+0000","last_fullsized":"2024-04-25T20:09:16.985459+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:58.259077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-25T20:09:18.631686+0000","last_change":"2024-04-25T20:09:17.121148+0000","last_active":"2024-04-25T20:09:18.631686+0000","last_peered":"2024-04-25T20:09:18.631686+0000","last_clean":"2024-04-25T20:09:18.631686+0000","last_became_active":"2024-04-25T20:09:17.120312+0000","last_became_peered":"2024-04-25T20:09:17.120312+0000","last_unstale":"2024-04-25T20:09:18.631686+0000","last_undegraded":"2024-04-25T20:09:18.631686+0000","last_fullsized":"2024-04-25T20:09:18.631686+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_clean_scrub_stamp":"2024-04-25T20:09:15.745353+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:48.383841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:17.118607+0000","last_change":"2024-04-25T20:09:15.772839+0000","last_active":"2024-04-25T20:09:17.118607+0000","last_peered":"2024-04-25T20:09:17.118607+0000","last_clean":"2024-04-25T20:09:17.118607+0000","last_became_active":"2024-04-25T20:09:15.772551+0000","last_became_peered":"2024-04-25T20:09:15.772551+0000","last_unstale":"2024-04-25T20:09:17.118607+0000","last_undegraded":"2024-04-25T20:09:17.118607+0000","last_fullsized":"2024-04-25T20:09:17.118607+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:55.101025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.981712+0000","last_change":"2024-04-25T20:09:16.141037+0000","last_active":"2024-04-25T20:09:16.981712+0000","last_peered":"2024-04-25T20:09:16.981712+0000","last_clean":"2024-04-25T20:09:16.981712+0000","last_became_active":"2024-04-25T20:09:16.140704+0000","last_became_peered":"2024-04-25T20:09:16.140704+0000","last_unstale":"2024-04-25T20:09:16.981712+0000","last_undegraded":"2024-04-25T20:09:16.981712+0000","last_fullsized":"2024-04-25T20:09:16.981712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:22.537137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.985574+0000","last_change":"2024-04-25T20:09:16.140930+0000","last_active":"2024-04-25T20:09:16.985574+0000","last_peered":"2024-04-25T20:09:16.985574+0000","last_clean":"2024-04-25T20:09:16.985574+0000","last_became_active":"2024-04-25T20:09:16.140617+0000","last_became_peered":"2024-04-25T20:09:16.140617+0000","last_unstale":"2024-04-25T20:09:16.985574+0000","last_undegraded":"2024-04-25T20:09:16.985574+0000","last_fullsized":"2024-04-25T20:09:16.985574+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:41:45.137982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.981700+0000","last_change":"2024-04-25T20:09:16.141064+0000","last_active":"2024-04-25T20:09:16.981700+0000","last_peered":"2024-04-25T20:09:16.981700+0000","last_clean":"2024-04-25T20:09:16.981700+0000","last_became_active":"2024-04-25T20:09:16.140731+0000","last_became_peered":"2024-04-25T20:09:16.140731+0000","last_unstale":"2024-04-25T20:09:16.981700+0000","last_undegraded":"2024-04-25T20:09:16.981700+0000","last_fullsized":"2024-04-25T20:09:16.981700+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:18.458296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":225280,"data_stored":643257,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":52889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":52889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":167936,"data_stored":52889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31284,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340556,"statfs":{"total":96636764160,"available":96604729344,"internally_reserved":0,"allocated":167936,"data_stored":52889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":167936,"data_stored":52889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":225280,"data_stored":643257,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":163840,"data_stored":51953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:09:25.322 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T20:09:25.611 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:09:25.611 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-04-25T20:09:25.623 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-04-25T20:09:25.604569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234064,"kb_used_data":1420,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754740656,"statfs":{"total":773094113280,"available":772854431744,"internally_reserved":0,"allocated":1454080,"data_stored":1602912,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001083"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.983771+0000","last_change":"2024-04-25T20:09:15.769542+0000","last_active":"2024-04-25T20:09:16.983771+0000","last_peered":"2024-04-25T20:09:16.983771+0000","last_clean":"2024-04-25T20:09:16.983771+0000","last_became_active":"2024-04-25T20:09:15.769271+0000","last_became_peered":"2024-04-25T20:09:15.769271+0000","last_unstale":"2024-04-25T20:09:16.983771+0000","last_undegraded":"2024-04-25T20:09:16.983771+0000","last_fullsized":"2024-04-25T20:09:16.983771+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:31.359804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.983757+0000","last_change":"2024-04-25T20:09:16.141209+0000","last_active":"2024-04-25T20:09:16.983757+0000","last_peered":"2024-04-25T20:09:16.983757+0000","last_clean":"2024-04-25T20:09:16.983757+0000","last_became_active":"2024-04-25T20:09:16.140616+0000","last_became_peered":"2024-04-25T20:09:16.140616+0000","last_unstale":"2024-04-25T20:09:16.983757+0000","last_undegraded":"2024-04-25T20:09:16.983757+0000","last_fullsized":"2024-04-25T20:09:16.983757+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:31.562175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.984751+0000","last_change":"2024-04-25T20:09:15.762535+0000","last_active":"2024-04-25T20:09:16.984751+0000","last_peered":"2024-04-25T20:09:16.984751+0000","last_clean":"2024-04-25T20:09:16.984751+0000","last_became_active":"2024-04-25T20:09:15.762247+0000","last_became_peered":"2024-04-25T20:09:15.762247+0000","last_unstale":"2024-04-25T20:09:16.984751+0000","last_undegraded":"2024-04-25T20:09:16.984751+0000","last_fullsized":"2024-04-25T20:09:16.984751+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:06:25.576044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.985459+0000","last_change":"2024-04-25T20:09:15.775252+0000","last_active":"2024-04-25T20:09:16.985459+0000","last_peered":"2024-04-25T20:09:16.985459+0000","last_clean":"2024-04-25T20:09:16.985459+0000","last_became_active":"2024-04-25T20:09:15.775027+0000","last_became_peered":"2024-04-25T20:09:15.775027+0000","last_unstale":"2024-04-25T20:09:16.985459+0000","last_undegraded":"2024-04-25T20:09:16.985459+0000","last_fullsized":"2024-04-25T20:09:16.985459+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:58.259077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-25T20:09:18.631686+0000","last_change":"2024-04-25T20:09:17.121148+0000","last_active":"2024-04-25T20:09:18.631686+0000","last_peered":"2024-04-25T20:09:18.631686+0000","last_clean":"2024-04-25T20:09:18.631686+0000","last_became_active":"2024-04-25T20:09:17.120312+0000","last_became_peered":"2024-04-25T20:09:17.120312+0000","last_unstale":"2024-04-25T20:09:18.631686+0000","last_undegraded":"2024-04-25T20:09:18.631686+0000","last_fullsized":"2024-04-25T20:09:18.631686+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_clean_scrub_stamp":"2024-04-25T20:09:15.745353+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:48.383841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:17.118607+0000","last_change":"2024-04-25T20:09:15.772839+0000","last_active":"2024-04-25T20:09:17.118607+0000","last_peered":"2024-04-25T20:09:17.118607+0000","last_clean":"2024-04-25T20:09:17.118607+0000","last_became_active":"2024-04-25T20:09:15.772551+0000","last_became_peered":"2024-04-25T20:09:15.772551+0000","last_unstale":"2024-04-25T20:09:17.118607+0000","last_undegraded":"2024-04-25T20:09:17.118607+0000","last_fullsized":"2024-04-25T20:09:17.118607+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:55.101025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.981712+0000","last_change":"2024-04-25T20:09:16.141037+0000","last_active":"2024-04-25T20:09:16.981712+0000","last_peered":"2024-04-25T20:09:16.981712+0000","last_clean":"2024-04-25T20:09:16.981712+0000","last_became_active":"2024-04-25T20:09:16.140704+0000","last_became_peered":"2024-04-25T20:09:16.140704+0000","last_unstale":"2024-04-25T20:09:16.981712+0000","last_undegraded":"2024-04-25T20:09:16.981712+0000","last_fullsized":"2024-04-25T20:09:16.981712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:22.537137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.985574+0000","last_change":"2024-04-25T20:09:16.140930+0000","last_active":"2024-04-25T20:09:16.985574+0000","last_peered":"2024-04-25T20:09:16.985574+0000","last_clean":"2024-04-25T20:09:16.985574+0000","last_became_active":"2024-04-25T20:09:16.140617+0000","last_became_peered":"2024-04-25T20:09:16.140617+0000","last_unstale":"2024-04-25T20:09:16.985574+0000","last_undegraded":"2024-04-25T20:09:16.985574+0000","last_fullsized":"2024-04-25T20:09:16.985574+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:41:45.137982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T20:09:16.981700+0000","last_change":"2024-04-25T20:09:16.141064+0000","last_active":"2024-04-25T20:09:16.981700+0000","last_peered":"2024-04-25T20:09:16.981700+0000","last_clean":"2024-04-25T20:09:16.981700+0000","last_became_active":"2024-04-25T20:09:16.140731+0000","last_became_peered":"2024-04-25T20:09:16.140731+0000","last_unstale":"2024-04-25T20:09:16.981700+0000","last_undegraded":"2024-04-25T20:09:16.981700+0000","last_fullsized":"2024-04-25T20:09:16.981700+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:18.458296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":225280,"data_stored":643257,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":52889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":52889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":167936,"data_stored":52889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31284,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340556,"statfs":{"total":96636764160,"available":96604729344,"internally_reserved":0,"allocated":167936,"data_stored":52889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":167936,"data_stored":52889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":225280,"data_stored":643257,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":163840,"data_stored":51953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:09:25.624 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T20:09:25.624 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-04-25T20:09:25.918 INFO:teuthology.orchestra.run.smithi039.stdout:epoch 17 2024-04-25T20:09:25.918 INFO:teuthology.orchestra.run.smithi039.stdout:fsid 975ceafd-555d-4f71-81e4-1f70364da00f 2024-04-25T20:09:25.918 INFO:teuthology.orchestra.run.smithi039.stdout:created 2024-04-25T20:09:03.377320+0000 2024-04-25T20:09:25.918 INFO:teuthology.orchestra.run.smithi039.stdout:modified 2024-04-25T20:09:17.770532+0000 2024-04-25T20:09:25.918 INFO:teuthology.orchestra.run.smithi039.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T20:09:25.918 INFO:teuthology.orchestra.run.smithi039.stdout:crush_version 4 2024-04-25T20:09:25.918 INFO:teuthology.orchestra.run.smithi039.stdout:full_ratio 0.95 2024-04-25T20:09:25.918 INFO:teuthology.orchestra.run.smithi039.stdout:backfillfull_ratio 0.9 2024-04-25T20:09:25.918 INFO:teuthology.orchestra.run.smithi039.stdout:nearfull_ratio 0.85 2024-04-25T20:09:25.918 INFO:teuthology.orchestra.run.smithi039.stdout:require_min_compat_client luminous 2024-04-25T20:09:25.919 INFO:teuthology.orchestra.run.smithi039.stdout:min_compat_client jewel 2024-04-25T20:09:25.919 INFO:teuthology.orchestra.run.smithi039.stdout:require_osd_release reef 2024-04-25T20:09:25.919 INFO:teuthology.orchestra.run.smithi039.stdout:stretch_mode_enabled false 2024-04-25T20:09:25.919 INFO:teuthology.orchestra.run.smithi039.stdout:pool 1 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 16 flags hashpspool stripe_width 0 application rbd read_balance_score 2.00 2024-04-25T20:09:25.919 INFO:teuthology.orchestra.run.smithi039.stdout:pool 2 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-04-25T20:09:25.919 INFO:teuthology.orchestra.run.smithi039.stdout:max_osd 8 2024-04-25T20:09:25.919 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6816/31832,v1:172.21.15.39:6818/31832] [v2:172.21.15.39:6820/31832,v1:172.21.15.39:6822/31832] exists,up 7a7118ba-4aee-404f-97e3-94c50e2996c4 2024-04-25T20:09:25.919 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6808/31831,v1:172.21.15.39:6809/31831] [v2:172.21.15.39:6810/31831,v1:172.21.15.39:6811/31831] exists,up 13637dbe-e37f-457f-b68b-8eed1b970617 2024-04-25T20:09:25.919 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6800/31829,v1:172.21.15.39:6801/31829] [v2:172.21.15.39:6802/31829,v1:172.21.15.39:6803/31829] exists,up 29e20ce2-4776-455d-8cf4-d539e738a6a7 2024-04-25T20:09:25.919 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6817/31830,v1:172.21.15.39:6819/31830] [v2:172.21.15.39:6821/31830,v1:172.21.15.39:6823/31830] exists,up a8b34e02-0898-4303-9fd5-62cbba9fec5c 2024-04-25T20:09:25.919 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6800/31724,v1:172.21.15.135:6801/31724] [v2:172.21.15.135:6802/31724,v1:172.21.15.135:6803/31724] exists,up e92b5c67-6908-4fd4-9940-5cb98b5ee847 2024-04-25T20:09:25.919 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6820/31734,v1:172.21.15.135:6822/31734] [v2:172.21.15.135:6824/31734,v1:172.21.15.135:6826/31734] exists,up d74d8fef-0211-413b-bff2-f97776fc1faa 2024-04-25T20:09:25.919 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6808/31725,v1:172.21.15.135:6809/31725] [v2:172.21.15.135:6810/31725,v1:172.21.15.135:6811/31725] exists,up 7dc84c6e-76d8-4f22-8b17-51f62765762d 2024-04-25T20:09:25.919 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6816/31737,v1:172.21.15.135:6817/31737] [v2:172.21.15.135:6818/31737,v1:172.21.15.135:6819/31737] exists,up 7adb3187-aed3-48af-868c-7230e9bfc85e 2024-04-25T20:09:25.931 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6816/31832,v1:172.21.15.39:6818/31832] [v2:172.21.15.39:6820/31832,v1:172.21.15.39:6822/31832] exists,up 7a7118ba-4aee-404f-97e3-94c50e2996c4', 'osd.1 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6808/31831,v1:172.21.15.39:6809/31831] [v2:172.21.15.39:6810/31831,v1:172.21.15.39:6811/31831] exists,up 13637dbe-e37f-457f-b68b-8eed1b970617', 'osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6800/31829,v1:172.21.15.39:6801/31829] [v2:172.21.15.39:6802/31829,v1:172.21.15.39:6803/31829] exists,up 29e20ce2-4776-455d-8cf4-d539e738a6a7', 'osd.3 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6817/31830,v1:172.21.15.39:6819/31830] [v2:172.21.15.39:6821/31830,v1:172.21.15.39:6823/31830] exists,up a8b34e02-0898-4303-9fd5-62cbba9fec5c', 'osd.4 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6800/31724,v1:172.21.15.135:6801/31724] [v2:172.21.15.135:6802/31724,v1:172.21.15.135:6803/31724] exists,up e92b5c67-6908-4fd4-9940-5cb98b5ee847', 'osd.5 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6820/31734,v1:172.21.15.135:6822/31734] [v2:172.21.15.135:6824/31734,v1:172.21.15.135:6826/31734] exists,up d74d8fef-0211-413b-bff2-f97776fc1faa', 'osd.6 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6808/31725,v1:172.21.15.135:6809/31725] [v2:172.21.15.135:6810/31725,v1:172.21.15.135:6811/31725] exists,up 7dc84c6e-76d8-4f22-8b17-51f62765762d', 'osd.7 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6816/31737,v1:172.21.15.135:6817/31737] [v2:172.21.15.135:6818/31737,v1:172.21.15.135:6819/31737] exists,up 7adb3187-aed3-48af-868c-7230e9bfc85e'] 2024-04-25T20:09:25.931 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-04-25T20:09:26.035 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:26.035 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mon.a", 2024-04-25T20:09:26.035 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster": "ceph", 2024-04-25T20:09:26.035 INFO:teuthology.orchestra.run.smithi039.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-04-25T20:09:26.035 INFO:teuthology.orchestra.run.smithi039.stdout: "admin_socket_mode": "", 2024-04-25T20:09:26.035 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-04-25T20:09:26.036 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_client_required": "cephx, none", 2024-04-25T20:09:26.036 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_cluster_required": "cephx", 2024-04-25T20:09:26.036 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_debug": "true", 2024-04-25T20:09:26.036 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-04-25T20:09:26.036 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-04-25T20:09:26.036 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_service_required": "cephx", 2024-04-25T20:09:26.036 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_service_ticket_ttl": "240.000000", 2024-04-25T20:09:26.036 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_supported": "cephx", 2024-04-25T20:09:26.036 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_aio": "true", 2024-04-25T20:09:26.036 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_aio_max_queue_depth": "1024", 2024-04-25T20:09:26.036 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_aio_poll_ms": "250", 2024-04-25T20:09:26.036 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_aio_reap_max": "16", 2024-04-25T20:09:26.036 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_async_discard": "false", 2024-04-25T20:09:26.036 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_block_size": "4096", 2024-04-25T20:09:26.037 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_debug_aio": "false", 2024-04-25T20:09:26.037 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-04-25T20:09:26.037 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-04-25T20:09:26.037 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_debug_inflight_ios": "false", 2024-04-25T20:09:26.037 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_enable_discard": "false", 2024-04-25T20:09:26.037 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_flock_retry": "3", 2024-04-25T20:09:26.037 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_flock_retry_interval": "0.100000", 2024-04-25T20:09:26.037 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_inject_crash": "0", 2024-04-25T20:09:26.037 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_inject_crash_flush_delay": "2", 2024-04-25T20:09:26.037 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_ioring": "false", 2024-04-25T20:09:26.037 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_ioring_hipri": "false", 2024-04-25T20:09:26.037 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_ioring_sqthread_poll": "false", 2024-04-25T20:09:26.037 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-04-25T20:09:26.037 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_read_buffer_alignment": "4096", 2024-04-25T20:09:26.038 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-04-25T20:09:26.038 INFO:teuthology.orchestra.run.smithi039.stdout: "bdev_type": "", 2024-04-25T20:09:26.038 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_alloc_size": "1048576", 2024-04-25T20:09:26.038 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_allocator": "hybrid", 2024-04-25T20:09:26.038 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_buffered_io": "true", 2024-04-25T20:09:26.038 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_check_for_zeros": "false", 2024-04-25T20:09:26.038 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_check_volume_selector_often": "false", 2024-04-25T20:09:26.038 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-04-25T20:09:26.038 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_compact_log_sync": "false", 2024-04-25T20:09:26.038 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-04-25T20:09:26.038 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-04-25T20:09:26.038 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_log_compact_min_size": "16777216", 2024-04-25T20:09:26.038 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_log_replay_check_allocations": "true", 2024-04-25T20:09:26.038 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_max_log_runway": "4194304", 2024-04-25T20:09:26.039 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_max_prefetch": "1048576", 2024-04-25T20:09:26.039 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_min_flush_size": "524288", 2024-04-25T20:09:26.039 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_min_log_runway": "1048576", 2024-04-25T20:09:26.039 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_replay_recovery": "false", 2024-04-25T20:09:26.039 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-04-25T20:09:26.039 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_shared_alloc_size": "65536", 2024-04-25T20:09:26.039 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_sync_write": "false", 2024-04-25T20:09:26.039 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-04-25T20:09:26.039 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-04-25T20:09:26.039 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-04-25T20:09:26.039 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_allocation_from_file": "true", 2024-04-25T20:09:26.039 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_allocator": "hybrid", 2024-04-25T20:09:26.039 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-04-25T20:09:26.039 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-04-25T20:09:26.040 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-04-25T20:09:26.040 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-04-25T20:09:26.040 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-04-25T20:09:26.040 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-04-25T20:09:26.040 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blobid_prealloc": "10240", 2024-04-25T20:09:26.040 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_block_create": "true", 2024-04-25T20:09:26.040 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_block_db_create": "false", 2024-04-25T20:09:26.040 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_block_db_path": "", 2024-04-25T20:09:26.040 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_block_db_size": "0", 2024-04-25T20:09:26.040 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_block_path": "", 2024-04-25T20:09:26.041 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_block_preallocate_file": "false", 2024-04-25T20:09:26.041 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_block_size": "107374182400", 2024-04-25T20:09:26.041 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_block_wal_create": "false", 2024-04-25T20:09:26.041 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_block_wal_path": "", 2024-04-25T20:09:26.041 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_block_wal_size": "100663296", 2024-04-25T20:09:26.041 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_bluefs": "true", 2024-04-25T20:09:26.041 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-04-25T20:09:26.041 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_bluefs_env_mirror": "false", 2024-04-25T20:09:26.041 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-04-25T20:09:26.041 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-04-25T20:09:26.041 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-04-25T20:09:26.041 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-04-25T20:09:26.041 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-04-25T20:09:26.041 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-04-25T20:09:26.042 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_autotune": "true", 2024-04-25T20:09:26.042 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-04-25T20:09:26.042 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-04-25T20:09:26.042 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-04-25T20:09:26.042 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-04-25T20:09:26.042 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_size": "0", 2024-04-25T20:09:26.042 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-04-25T20:09:26.042 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-04-25T20:09:26.042 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-04-25T20:09:26.042 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-04-25T20:09:26.042 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_type": "2q", 2024-04-25T20:09:26.042 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-04-25T20:09:26.042 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_clone_cow": "true", 2024-04-25T20:09:26.042 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_compression_algorithm": "snappy", 2024-04-25T20:09:26.043 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_compression_max_blob_size": "0", 2024-04-25T20:09:26.043 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-04-25T20:09:26.043 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-04-25T20:09:26.043 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_compression_min_blob_size": "0", 2024-04-25T20:09:26.043 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-04-25T20:09:26.043 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-04-25T20:09:26.043 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_compression_mode": "none", 2024-04-25T20:09:26.043 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-04-25T20:09:26.043 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_csum_type": "crc32c", 2024-04-25T20:09:26.043 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_enforce_settings": "default", 2024-04-25T20:09:26.043 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_freelist": "false", 2024-04-25T20:09:26.043 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_fsck_abort": "false", 2024-04-25T20:09:26.043 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-04-25T20:09:26.043 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_inject_bug21040": "false", 2024-04-25T20:09:26.043 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-04-25T20:09:26.044 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_inject_read_err": "false", 2024-04-25T20:09:26.044 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_legacy_omap": "false", 2024-04-25T20:09:26.044 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-04-25T20:09:26.044 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-04-25T20:09:26.044 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-04-25T20:09:26.044 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-04-25T20:09:26.044 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_prefill": "0.000000", 2024-04-25T20:09:26.044 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-04-25T20:09:26.044 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-04-25T20:09:26.044 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-04-25T20:09:26.044 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_small_allocations": "0", 2024-04-25T20:09:26.044 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-04-25T20:09:26.044 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_default_buffered_read": "true", 2024-04-25T20:09:26.044 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_default_buffered_write": "false", 2024-04-25T20:09:26.044 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_deferred_batch_ops": "0", 2024-04-25T20:09:26.044 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-04-25T20:09:26.045 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-04-25T20:09:26.045 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-04-25T20:09:26.045 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-04-25T20:09:26.045 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-04-25T20:09:26.045 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-04-25T20:09:26.045 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-04-25T20:09:26.045 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fail_eio": "false", 2024-04-25T20:09:26.045 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-04-25T20:09:26.045 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-04-25T20:09:26.045 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-04-25T20:09:26.045 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-04-25T20:09:26.045 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_on_mkfs": "true", 2024-04-25T20:09:26.045 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-04-25T20:09:26.045 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_on_mount": "false", 2024-04-25T20:09:26.045 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-04-25T20:09:26.046 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_on_umount": "false", 2024-04-25T20:09:26.046 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-04-25T20:09:26.046 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-04-25T20:09:26.046 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-04-25T20:09:26.046 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-04-25T20:09:26.046 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-04-25T20:09:26.046 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-04-25T20:09:26.046 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-04-25T20:09:26.046 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-04-25T20:09:26.046 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_ignore_data_csum": "false", 2024-04-25T20:09:26.046 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-04-25T20:09:26.046 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_kvbackend": "rocksdb", 2024-04-25T20:09:26.046 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-04-25T20:09:26.046 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-04-25T20:09:26.047 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_log_op_age": "5.000000", 2024-04-25T20:09:26.047 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_max_alloc_size": "0", 2024-04-25T20:09:26.047 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_max_blob_size": "0", 2024-04-25T20:09:26.047 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-04-25T20:09:26.047 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-04-25T20:09:26.047 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_max_defer_interval": "3.000000", 2024-04-25T20:09:26.047 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_max_deferred_txc": "32", 2024-04-25T20:09:26.047 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_min_alloc_size": "0", 2024-04-25T20:09:26.047 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-04-25T20:09:26.047 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-04-25T20:09:26.047 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_nid_prealloc": "1024", 2024-04-25T20:09:26.047 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_prefer_deferred_size": "0", 2024-04-25T20:09:26.047 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-04-25T20:09:26.047 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-04-25T20:09:26.047 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_qfsck_on_mount": "true", 2024-04-25T20:09:26.048 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_retry_disk_reads": "3", 2024-04-25T20:09:26.048 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_rocksdb_cf": "true", 2024-04-25T20:09:26.048 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-04-25T20:09:26.048 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-04-25T20:09:26.048 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_rocksdb_options_annex": "", 2024-04-25T20:09:26.048 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_spdk_coremask": "0x1", 2024-04-25T20:09:26.048 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_spdk_io_sleep": "5", 2024-04-25T20:09:26.048 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_spdk_max_io_completion": "0", 2024-04-25T20:09:26.048 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_spdk_mem": "512", 2024-04-25T20:09:26.048 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_sync_submit_transaction": "false", 2024-04-25T20:09:26.048 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_throttle_bytes": "67108864", 2024-04-25T20:09:26.048 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_throttle_cost_per_io": "0", 2024-04-25T20:09:26.048 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-04-25T20:09:26.049 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-04-25T20:09:26.049 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-04-25T20:09:26.049 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-04-25T20:09:26.049 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_tracing": "false", 2024-04-25T20:09:26.049 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-04-25T20:09:26.049 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-04-25T20:09:26.049 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_volume_selection_reserved": "0", 2024-04-25T20:09:26.049 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-04-25T20:09:26.049 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-04-25T20:09:26.049 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-04-25T20:09:26.049 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-04-25T20:09:26.049 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-04-25T20:09:26.049 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-04-25T20:09:26.049 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_zero_block_detection": "false", 2024-04-25T20:09:26.049 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-04-25T20:09:26.050 INFO:teuthology.orchestra.run.smithi039.stdout: "cephfs_mirror_action_update_interval": "2", 2024-04-25T20:09:26.050 INFO:teuthology.orchestra.run.smithi039.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-04-25T20:09:26.050 INFO:teuthology.orchestra.run.smithi039.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-04-25T20:09:26.050 INFO:teuthology.orchestra.run.smithi039.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-04-25T20:09:26.050 INFO:teuthology.orchestra.run.smithi039.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-04-25T20:09:26.050 INFO:teuthology.orchestra.run.smithi039.stdout: "cephfs_mirror_mount_timeout": "10", 2024-04-25T20:09:26.050 INFO:teuthology.orchestra.run.smithi039.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-04-25T20:09:26.050 INFO:teuthology.orchestra.run.smithi039.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-04-25T20:09:26.050 INFO:teuthology.orchestra.run.smithi039.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-04-25T20:09:26.050 INFO:teuthology.orchestra.run.smithi039.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-04-25T20:09:26.050 INFO:teuthology.orchestra.run.smithi039.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-04-25T20:09:26.050 INFO:teuthology.orchestra.run.smithi039.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-04-25T20:09:26.050 INFO:teuthology.orchestra.run.smithi039.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-04-25T20:09:26.050 INFO:teuthology.orchestra.run.smithi039.stdout: "cephx_cluster_require_signatures": "false", 2024-04-25T20:09:26.051 INFO:teuthology.orchestra.run.smithi039.stdout: "cephx_cluster_require_version": "2", 2024-04-25T20:09:26.051 INFO:teuthology.orchestra.run.smithi039.stdout: "cephx_require_signatures": "false", 2024-04-25T20:09:26.051 INFO:teuthology.orchestra.run.smithi039.stdout: "cephx_require_version": "2", 2024-04-25T20:09:26.051 INFO:teuthology.orchestra.run.smithi039.stdout: "cephx_service_require_signatures": "false", 2024-04-25T20:09:26.051 INFO:teuthology.orchestra.run.smithi039.stdout: "cephx_service_require_version": "2", 2024-04-25T20:09:26.051 INFO:teuthology.orchestra.run.smithi039.stdout: "cephx_sign_messages": "true", 2024-04-25T20:09:26.051 INFO:teuthology.orchestra.run.smithi039.stdout: "chdir": "", 2024-04-25T20:09:26.051 INFO:teuthology.orchestra.run.smithi039.stdout: "client_acl_type": "", 2024-04-25T20:09:26.051 INFO:teuthology.orchestra.run.smithi039.stdout: "client_asio_thread_count": "2", 2024-04-25T20:09:26.051 INFO:teuthology.orchestra.run.smithi039.stdout: "client_cache_mid": "0.750000", 2024-04-25T20:09:26.052 INFO:teuthology.orchestra.run.smithi039.stdout: "client_cache_size": "16384", 2024-04-25T20:09:26.052 INFO:teuthology.orchestra.run.smithi039.stdout: "client_caps_release_delay": "5", 2024-04-25T20:09:26.052 INFO:teuthology.orchestra.run.smithi039.stdout: "client_check_pool_perm": "true", 2024-04-25T20:09:26.052 INFO:teuthology.orchestra.run.smithi039.stdout: "client_collect_and_send_global_metrics": "false", 2024-04-25T20:09:26.052 INFO:teuthology.orchestra.run.smithi039.stdout: "client_debug_force_sync_read": "false", 2024-04-25T20:09:26.052 INFO:teuthology.orchestra.run.smithi039.stdout: "client_debug_getattr_caps": "false", 2024-04-25T20:09:26.052 INFO:teuthology.orchestra.run.smithi039.stdout: "client_debug_inject_tick_delay": "0", 2024-04-25T20:09:26.052 INFO:teuthology.orchestra.run.smithi039.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-04-25T20:09:26.052 INFO:teuthology.orchestra.run.smithi039.stdout: "client_die_on_failed_remount": "false", 2024-04-25T20:09:26.052 INFO:teuthology.orchestra.run.smithi039.stdout: "client_dirsize_rbytes": "true", 2024-04-25T20:09:26.052 INFO:teuthology.orchestra.run.smithi039.stdout: "client_force_lazyio": "false", 2024-04-25T20:09:26.052 INFO:teuthology.orchestra.run.smithi039.stdout: "client_fs": "", 2024-04-25T20:09:26.052 INFO:teuthology.orchestra.run.smithi039.stdout: "client_inject_fixed_oldest_tid": "false", 2024-04-25T20:09:26.053 INFO:teuthology.orchestra.run.smithi039.stdout: "client_inject_release_failure": "false", 2024-04-25T20:09:26.053 INFO:teuthology.orchestra.run.smithi039.stdout: "client_max_inline_size": "4096", 2024-04-25T20:09:26.053 INFO:teuthology.orchestra.run.smithi039.stdout: "client_max_retries_on_remount_failure": "5", 2024-04-25T20:09:26.053 INFO:teuthology.orchestra.run.smithi039.stdout: "client_mds_namespace": "", 2024-04-25T20:09:26.053 INFO:teuthology.orchestra.run.smithi039.stdout: "client_metadata": "", 2024-04-25T20:09:26.053 INFO:teuthology.orchestra.run.smithi039.stdout: "client_mount_gid": "-1", 2024-04-25T20:09:26.053 INFO:teuthology.orchestra.run.smithi039.stdout: "client_mount_timeout": "300", 2024-04-25T20:09:26.053 INFO:teuthology.orchestra.run.smithi039.stdout: "client_mount_uid": "-1", 2024-04-25T20:09:26.053 INFO:teuthology.orchestra.run.smithi039.stdout: "client_mountpoint": "/", 2024-04-25T20:09:26.053 INFO:teuthology.orchestra.run.smithi039.stdout: "client_notify_timeout": "10", 2024-04-25T20:09:26.053 INFO:teuthology.orchestra.run.smithi039.stdout: "client_oc": "true", 2024-04-25T20:09:26.053 INFO:teuthology.orchestra.run.smithi039.stdout: "client_oc_max_dirty": "104857600", 2024-04-25T20:09:26.053 INFO:teuthology.orchestra.run.smithi039.stdout: "client_oc_max_dirty_age": "5.000000", 2024-04-25T20:09:26.053 INFO:teuthology.orchestra.run.smithi039.stdout: "client_oc_max_objects": "1000", 2024-04-25T20:09:26.054 INFO:teuthology.orchestra.run.smithi039.stdout: "client_oc_size": "209715200", 2024-04-25T20:09:26.054 INFO:teuthology.orchestra.run.smithi039.stdout: "client_oc_target_dirty": "8388608", 2024-04-25T20:09:26.054 INFO:teuthology.orchestra.run.smithi039.stdout: "client_permissions": "true", 2024-04-25T20:09:26.054 INFO:teuthology.orchestra.run.smithi039.stdout: "client_quota": "true", 2024-04-25T20:09:26.054 INFO:teuthology.orchestra.run.smithi039.stdout: "client_quota_df": "true", 2024-04-25T20:09:26.054 INFO:teuthology.orchestra.run.smithi039.stdout: "client_readahead_max_bytes": "0", 2024-04-25T20:09:26.054 INFO:teuthology.orchestra.run.smithi039.stdout: "client_readahead_max_periods": "4", 2024-04-25T20:09:26.054 INFO:teuthology.orchestra.run.smithi039.stdout: "client_readahead_min": "131072", 2024-04-25T20:09:26.054 INFO:teuthology.orchestra.run.smithi039.stdout: "client_reconnect_stale": "false", 2024-04-25T20:09:26.054 INFO:teuthology.orchestra.run.smithi039.stdout: "client_shutdown_timeout": "30", 2024-04-25T20:09:26.054 INFO:teuthology.orchestra.run.smithi039.stdout: "client_snapdir": ".snap", 2024-04-25T20:09:26.054 INFO:teuthology.orchestra.run.smithi039.stdout: "client_tick_interval": "1", 2024-04-25T20:09:26.054 INFO:teuthology.orchestra.run.smithi039.stdout: "client_trace": "", 2024-04-25T20:09:26.054 INFO:teuthology.orchestra.run.smithi039.stdout: "client_try_dentry_invalidate": "false", 2024-04-25T20:09:26.055 INFO:teuthology.orchestra.run.smithi039.stdout: "client_use_faked_inos": "false", 2024-04-25T20:09:26.055 INFO:teuthology.orchestra.run.smithi039.stdout: "client_use_random_mds": "false", 2024-04-25T20:09:26.055 INFO:teuthology.orchestra.run.smithi039.stdout: "clog_to_graylog": "false", 2024-04-25T20:09:26.055 INFO:teuthology.orchestra.run.smithi039.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-04-25T20:09:26.055 INFO:teuthology.orchestra.run.smithi039.stdout: "clog_to_graylog_port": "12201", 2024-04-25T20:09:26.055 INFO:teuthology.orchestra.run.smithi039.stdout: "clog_to_monitors": "default=true", 2024-04-25T20:09:26.055 INFO:teuthology.orchestra.run.smithi039.stdout: "clog_to_syslog": "false", 2024-04-25T20:09:26.055 INFO:teuthology.orchestra.run.smithi039.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-04-25T20:09:26.055 INFO:teuthology.orchestra.run.smithi039.stdout: "clog_to_syslog_level": "info", 2024-04-25T20:09:26.055 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "-", 2024-04-25T20:09:26.055 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_network": "", 2024-04-25T20:09:26.055 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_network_interface": "", 2024-04-25T20:09:26.055 INFO:teuthology.orchestra.run.smithi039.stdout: "compressor_zlib_isal": "false", 2024-04-25T20:09:26.055 INFO:teuthology.orchestra.run.smithi039.stdout: "compressor_zlib_level": "5", 2024-04-25T20:09:26.056 INFO:teuthology.orchestra.run.smithi039.stdout: "compressor_zlib_winsize": "-15", 2024-04-25T20:09:26.056 INFO:teuthology.orchestra.run.smithi039.stdout: "compressor_zstd_level": "1", 2024-04-25T20:09:26.056 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-04-25T20:09:26.056 INFO:teuthology.orchestra.run.smithi039.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-04-25T20:09:26.056 INFO:teuthology.orchestra.run.smithi039.stdout: "crimson_alien_op_num_threads": "6", 2024-04-25T20:09:26.056 INFO:teuthology.orchestra.run.smithi039.stdout: "crimson_alien_thread_cpu_cores": "", 2024-04-25T20:09:26.056 INFO:teuthology.orchestra.run.smithi039.stdout: "crimson_osd_obc_lru_size": "10", 2024-04-25T20:09:26.056 INFO:teuthology.orchestra.run.smithi039.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-04-25T20:09:26.056 INFO:teuthology.orchestra.run.smithi039.stdout: "crimson_seastar_smp": "1", 2024-04-25T20:09:26.056 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_location": "", 2024-04-25T20:09:26.056 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_location_hook": "", 2024-04-25T20:09:26.056 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_location_hook_timeout": "10", 2024-04-25T20:09:26.056 INFO:teuthology.orchestra.run.smithi039.stdout: "daemonize": "false", 2024-04-25T20:09:26.056 INFO:teuthology.orchestra.run.smithi039.stdout: "daos_pool": "tank", 2024-04-25T20:09:26.057 INFO:teuthology.orchestra.run.smithi039.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-04-25T20:09:26.057 INFO:teuthology.orchestra.run.smithi039.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-04-25T20:09:26.057 INFO:teuthology.orchestra.run.smithi039.stdout: "dbstore_db_name_prefix": "dbstore", 2024-04-25T20:09:26.057 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_alienstore": "0/5", 2024-04-25T20:09:26.057 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_allow_any_pool_priority": "false", 2024-04-25T20:09:26.057 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_asok": "1/5", 2024-04-25T20:09:26.057 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_asok_assert_abort": "false", 2024-04-25T20:09:26.057 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_asserts_on_shutdown": "true", 2024-04-25T20:09:26.057 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_auth": "20/20", 2024-04-25T20:09:26.057 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_bdev": "1/3", 2024-04-25T20:09:26.057 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_bluefs": "1/5", 2024-04-25T20:09:26.057 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_bluestore": "1/5", 2024-04-25T20:09:26.057 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_buffer": "0/1", 2024-04-25T20:09:26.057 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_ceph_exporter": "1/5", 2024-04-25T20:09:26.058 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_cephfs_mirror": "0/5", 2024-04-25T20:09:26.058 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_cephsqlite": "0/5", 2024-04-25T20:09:26.058 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_client": "0/5", 2024-04-25T20:09:26.058 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_compressor": "1/5", 2024-04-25T20:09:26.058 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_context": "0/1", 2024-04-25T20:09:26.058 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_crush": "1/1", 2024-04-25T20:09:26.058 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_crypto": "1/5", 2024-04-25T20:09:26.058 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_cyanstore": "0/5", 2024-04-25T20:09:26.058 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_deliberately_leak_memory": "false", 2024-04-25T20:09:26.058 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_disable_randomized_ping": "false", 2024-04-25T20:09:26.058 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_dpdk": "1/5", 2024-04-25T20:09:26.058 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_eventtrace": "1/5", 2024-04-25T20:09:26.058 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_filer": "0/1", 2024-04-25T20:09:26.058 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_filestore": "1/3", 2024-04-25T20:09:26.059 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_finisher": "1/1", 2024-04-25T20:09:26.059 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_fuse": "1/5", 2024-04-25T20:09:26.059 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_heartbeat_testing_span": "0", 2024-04-25T20:09:26.059 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_heartbeatmap": "1/5", 2024-04-25T20:09:26.059 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_immutable_obj_cache": "0/5", 2024-04-25T20:09:26.059 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_javaclient": "1/5", 2024-04-25T20:09:26.059 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_journal": "1/3", 2024-04-25T20:09:26.059 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_journaler": "0/5", 2024-04-25T20:09:26.059 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_kstore": "1/5", 2024-04-25T20:09:26.059 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_leveldb": "4/5", 2024-04-25T20:09:26.059 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_lockdep": "0/1", 2024-04-25T20:09:26.059 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_mclock": "1/5", 2024-04-25T20:09:26.059 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_mds": "1/5", 2024-04-25T20:09:26.059 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_mds_balancer": "1/5", 2024-04-25T20:09:26.059 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_mds_locker": "1/5", 2024-04-25T20:09:26.060 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_mds_log": "1/5", 2024-04-25T20:09:26.060 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_mds_log_expire": "1/5", 2024-04-25T20:09:26.060 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_mds_migrator": "1/5", 2024-04-25T20:09:26.060 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_memstore": "1/5", 2024-04-25T20:09:26.060 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_mgr": "2/5", 2024-04-25T20:09:26.060 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_mgrc": "1/5", 2024-04-25T20:09:26.060 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_mon": "20/20", 2024-04-25T20:09:26.060 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_monc": "0/10", 2024-04-25T20:09:26.060 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_ms": "1/1", 2024-04-25T20:09:26.060 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_none": "0/5", 2024-04-25T20:09:26.060 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_objclass": "0/5", 2024-04-25T20:09:26.060 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_objectcacher": "0/5", 2024-04-25T20:09:26.060 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_objecter": "0/1", 2024-04-25T20:09:26.060 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_optracker": "0/5", 2024-04-25T20:09:26.061 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_osd": "1/5", 2024-04-25T20:09:26.061 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_paxos": "20/20", 2024-04-25T20:09:26.061 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_perfcounter": "1/5", 2024-04-25T20:09:26.061 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_prioritycache": "1/5", 2024-04-25T20:09:26.061 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_rados": "0/5", 2024-04-25T20:09:26.061 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_rbd": "0/5", 2024-04-25T20:09:26.061 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_rbd_mirror": "0/5", 2024-04-25T20:09:26.061 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_rbd_pwl": "0/5", 2024-04-25T20:09:26.061 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_rbd_replay": "0/5", 2024-04-25T20:09:26.061 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_refs": "0/0", 2024-04-25T20:09:26.061 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_reserver": "1/1", 2024-04-25T20:09:26.061 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_rgw": "1/5", 2024-04-25T20:09:26.061 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_rgw_access": "1/5", 2024-04-25T20:09:26.061 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_rgw_datacache": "1/5", 2024-04-25T20:09:26.061 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_rgw_dbstore": "1/5", 2024-04-25T20:09:26.062 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_rgw_flight": "1/5", 2024-04-25T20:09:26.062 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_rgw_sync": "1/5", 2024-04-25T20:09:26.062 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_rocksdb": "4/5", 2024-04-25T20:09:26.062 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_seastore": "0/5", 2024-04-25T20:09:26.062 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_seastore_backref": "0/5", 2024-04-25T20:09:26.062 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_seastore_cache": "0/5", 2024-04-25T20:09:26.062 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_seastore_cleaner": "0/5", 2024-04-25T20:09:26.062 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_seastore_device": "0/5", 2024-04-25T20:09:26.062 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_seastore_epm": "0/5", 2024-04-25T20:09:26.062 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-04-25T20:09:26.062 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_seastore_journal": "0/5", 2024-04-25T20:09:26.062 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_seastore_lba": "0/5", 2024-04-25T20:09:26.062 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_seastore_odata": "0/5", 2024-04-25T20:09:26.063 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_seastore_omap": "0/5", 2024-04-25T20:09:26.063 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_seastore_onode": "0/5", 2024-04-25T20:09:26.063 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_seastore_t": "0/5", 2024-04-25T20:09:26.063 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_seastore_tm": "0/5", 2024-04-25T20:09:26.063 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_striper": "0/1", 2024-04-25T20:09:26.063 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_test": "0/5", 2024-04-25T20:09:26.063 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_throttle": "1/1", 2024-04-25T20:09:26.063 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_timer": "0/1", 2024-04-25T20:09:26.063 INFO:teuthology.orchestra.run.smithi039.stdout: "debug_tp": "0/5", 2024-04-25T20:09:26.063 INFO:teuthology.orchestra.run.smithi039.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-04-25T20:09:26.063 INFO:teuthology.orchestra.run.smithi039.stdout: "device_failure_prediction_mode": "none", 2024-04-25T20:09:26.063 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-04-25T20:09:26.063 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-04-25T20:09:26.063 INFO:teuthology.orchestra.run.smithi039.stdout: "err_to_graylog": "false", 2024-04-25T20:09:26.064 INFO:teuthology.orchestra.run.smithi039.stdout: "err_to_journald": "false", 2024-04-25T20:09:26.064 INFO:teuthology.orchestra.run.smithi039.stdout: "err_to_stderr": "true", 2024-04-25T20:09:26.064 INFO:teuthology.orchestra.run.smithi039.stdout: "err_to_syslog": "false", 2024-04-25T20:09:26.064 INFO:teuthology.orchestra.run.smithi039.stdout: "event_tracing": "false", 2024-04-25T20:09:26.064 INFO:teuthology.orchestra.run.smithi039.stdout: "exporter_addr": "0.0.0.0", 2024-04-25T20:09:26.064 INFO:teuthology.orchestra.run.smithi039.stdout: "exporter_http_port": "9926", 2024-04-25T20:09:26.064 INFO:teuthology.orchestra.run.smithi039.stdout: "exporter_prio_limit": "5", 2024-04-25T20:09:26.064 INFO:teuthology.orchestra.run.smithi039.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-04-25T20:09:26.064 INFO:teuthology.orchestra.run.smithi039.stdout: "exporter_sort_metrics": "true", 2024-04-25T20:09:26.064 INFO:teuthology.orchestra.run.smithi039.stdout: "exporter_stats_period": "5", 2024-04-25T20:09:26.064 INFO:teuthology.orchestra.run.smithi039.stdout: "fake_statfs_for_testing": "0", 2024-04-25T20:09:26.064 INFO:teuthology.orchestra.run.smithi039.stdout: "fatal_signal_handlers": "true", 2024-04-25T20:09:26.064 INFO:teuthology.orchestra.run.smithi039.stdout: "filer_max_purge_ops": "10", 2024-04-25T20:09:26.064 INFO:teuthology.orchestra.run.smithi039.stdout: "filer_max_truncate_ops": "128", 2024-04-25T20:09:26.064 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_apply_finisher_threads": "1", 2024-04-25T20:09:26.065 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_blackhole": "false", 2024-04-25T20:09:26.065 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_btrfs_clone_range": "true", 2024-04-25T20:09:26.065 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_btrfs_snap": "true", 2024-04-25T20:09:26.065 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_caller_concurrency": "10", 2024-04-25T20:09:26.065 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_collect_device_partition_information": "true", 2024-04-25T20:09:26.065 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_commit_timeout": "600.000000", 2024-04-25T20:09:26.065 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_debug_inject_read_err": "false", 2024-04-25T20:09:26.065 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_debug_omap_check": "false", 2024-04-25T20:09:26.065 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_debug_random_read_err": "0.000000", 2024-04-25T20:09:26.065 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_debug_verify_split": "false", 2024-04-25T20:09:26.065 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_dump_file": "", 2024-04-25T20:09:26.065 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-04-25T20:09:26.065 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-04-25T20:09:26.065 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_fadvise": "true", 2024-04-25T20:09:26.065 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_fail_eio": "true", 2024-04-25T20:09:26.066 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_fd_cache_shards": "16", 2024-04-25T20:09:26.066 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_fd_cache_size": "128", 2024-04-25T20:09:26.066 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_fiemap": "false", 2024-04-25T20:09:26.066 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_fiemap_threshold": "4096", 2024-04-25T20:09:26.066 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-04-25T20:09:26.066 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_index_retry_probability": "0.000000", 2024-04-25T20:09:26.066 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_inject_stall": "0", 2024-04-25T20:09:26.066 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_journal_parallel": "false", 2024-04-25T20:09:26.066 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_journal_trailing": "false", 2024-04-25T20:09:26.066 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_journal_writeahead": "false", 2024-04-25T20:09:26.066 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_kill_at": "0", 2024-04-25T20:09:26.066 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-04-25T20:09:26.066 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_max_inline_xattr_size": "0", 2024-04-25T20:09:26.067 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-04-25T20:09:26.067 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-04-25T20:09:26.067 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-04-25T20:09:26.067 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_max_inline_xattrs": "0", 2024-04-25T20:09:26.067 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-04-25T20:09:26.067 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_max_inline_xattrs_other": "2", 2024-04-25T20:09:26.067 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-04-25T20:09:26.067 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_max_sync_interval": "5.000000", 2024-04-25T20:09:26.067 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_max_xattr_value_size": "0", 2024-04-25T20:09:26.067 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-04-25T20:09:26.067 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-04-25T20:09:26.067 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-04-25T20:09:26.067 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_merge_threshold": "-10", 2024-04-25T20:09:26.067 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_min_sync_interval": "0.010000", 2024-04-25T20:09:26.068 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_odsync_write": "false", 2024-04-25T20:09:26.068 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_omap_backend": "rocksdb", 2024-04-25T20:09:26.068 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_omap_backend_path": "", 2024-04-25T20:09:26.068 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_omap_header_cache_size": "1024", 2024-04-25T20:09:26.068 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_ondisk_finisher_threads": "1", 2024-04-25T20:09:26.068 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-04-25T20:09:26.068 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_op_thread_timeout": "60", 2024-04-25T20:09:26.068 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_op_threads": "2", 2024-04-25T20:09:26.068 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_punch_hole": "false", 2024-04-25T20:09:26.068 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-04-25T20:09:26.068 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-04-25T20:09:26.069 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-04-25T20:09:26.069 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-04-25T20:09:26.069 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-04-25T20:09:26.069 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_queue_max_bytes": "104857600", 2024-04-25T20:09:26.069 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-04-25T20:09:26.069 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-04-25T20:09:26.069 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-04-25T20:09:26.069 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_queue_max_ops": "50", 2024-04-25T20:09:26.069 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-04-25T20:09:26.069 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_seek_data_hole": "false", 2024-04-25T20:09:26.069 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_sloppy_crc": "false", 2024-04-25T20:09:26.069 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-04-25T20:09:26.070 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_splice": "false", 2024-04-25T20:09:26.070 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_split_multiple": "2", 2024-04-25T20:09:26.070 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_split_rand_factor": "20", 2024-04-25T20:09:26.070 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_update_to": "1000", 2024-04-25T20:09:26.070 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-04-25T20:09:26.070 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-04-25T20:09:26.070 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-04-25T20:09:26.070 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-04-25T20:09:26.070 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-04-25T20:09:26.070 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-04-25T20:09:26.070 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_wbthrottle_enable": "true", 2024-04-25T20:09:26.070 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-04-25T20:09:26.070 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-04-25T20:09:26.070 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-04-25T20:09:26.071 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-04-25T20:09:26.071 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-04-25T20:09:26.071 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-04-25T20:09:26.071 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_xfs_extsize": "false", 2024-04-25T20:09:26.071 INFO:teuthology.orchestra.run.smithi039.stdout: "filestore_zfs_snap": "false", 2024-04-25T20:09:26.071 INFO:teuthology.orchestra.run.smithi039.stdout: "fio_dir": "/tmp/fio", 2024-04-25T20:09:26.071 INFO:teuthology.orchestra.run.smithi039.stdout: "fsid": "975ceafd-555d-4f71-81e4-1f70364da00f", 2024-04-25T20:09:26.071 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_allow_other": "true", 2024-04-25T20:09:26.071 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_atomic_o_trunc": "true", 2024-04-25T20:09:26.071 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_big_writes": "true", 2024-04-25T20:09:26.071 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_debug": "false", 2024-04-25T20:09:26.071 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_default_permissions": "false", 2024-04-25T20:09:26.071 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_disable_pagecache": "false", 2024-04-25T20:09:26.071 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_max_write": "0", 2024-04-25T20:09:26.071 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_multithreaded": "true", 2024-04-25T20:09:26.071 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_require_active_mds": "true", 2024-04-25T20:09:26.072 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_set_user_groups": "true", 2024-04-25T20:09:26.072 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_splice_move": "true", 2024-04-25T20:09:26.072 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_splice_read": "true", 2024-04-25T20:09:26.072 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_splice_write": "true", 2024-04-25T20:09:26.072 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_syncfs_on_mksnap": "true", 2024-04-25T20:09:26.072 INFO:teuthology.orchestra.run.smithi039.stdout: "fuse_use_invalidate_cb": "true", 2024-04-25T20:09:26.072 INFO:teuthology.orchestra.run.smithi039.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-04-25T20:09:26.072 INFO:teuthology.orchestra.run.smithi039.stdout: "gss_target_name": "ceph", 2024-04-25T20:09:26.072 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_file": "", 2024-04-25T20:09:26.072 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_inject_failure": "0", 2024-04-25T20:09:26.072 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_interval": "5", 2024-04-25T20:09:26.072 INFO:teuthology.orchestra.run.smithi039.stdout: "host": "smithi039", 2024-04-25T20:09:26.072 INFO:teuthology.orchestra.run.smithi039.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-04-25T20:09:26.072 INFO:teuthology.orchestra.run.smithi039.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-04-25T20:09:26.072 INFO:teuthology.orchestra.run.smithi039.stdout: "immutable_object_cache_max_size": "1073741824", 2024-04-25T20:09:26.073 INFO:teuthology.orchestra.run.smithi039.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-04-25T20:09:26.073 INFO:teuthology.orchestra.run.smithi039.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-04-25T20:09:26.073 INFO:teuthology.orchestra.run.smithi039.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-04-25T20:09:26.073 INFO:teuthology.orchestra.run.smithi039.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-04-25T20:09:26.073 INFO:teuthology.orchestra.run.smithi039.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-04-25T20:09:26.073 INFO:teuthology.orchestra.run.smithi039.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-04-25T20:09:26.073 INFO:teuthology.orchestra.run.smithi039.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-04-25T20:09:26.073 INFO:teuthology.orchestra.run.smithi039.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-04-25T20:09:26.073 INFO:teuthology.orchestra.run.smithi039.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-04-25T20:09:26.073 INFO:teuthology.orchestra.run.smithi039.stdout: "immutable_object_cache_watermark": "0.900000", 2024-04-25T20:09:26.073 INFO:teuthology.orchestra.run.smithi039.stdout: "inject_early_sigterm": "false", 2024-04-25T20:09:26.073 INFO:teuthology.orchestra.run.smithi039.stdout: "jaeger_agent_port": "6799", 2024-04-25T20:09:26.073 INFO:teuthology.orchestra.run.smithi039.stdout: "jaeger_tracing_enable": "false", 2024-04-25T20:09:26.073 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_aio": "true", 2024-04-25T20:09:26.073 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_align_min_size": "65536", 2024-04-25T20:09:26.074 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_block_align": "true", 2024-04-25T20:09:26.074 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_block_size": "4096", 2024-04-25T20:09:26.074 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_dio": "true", 2024-04-25T20:09:26.074 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_discard": "false", 2024-04-25T20:09:26.074 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_force_aio": "false", 2024-04-25T20:09:26.074 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_ignore_corruption": "false", 2024-04-25T20:09:26.074 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_max_write_bytes": "10485760", 2024-04-25T20:09:26.074 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_max_write_entries": "100", 2024-04-25T20:09:26.074 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_replay_from": "0", 2024-04-25T20:09:26.074 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_throttle_high_multiple": "0.000000", 2024-04-25T20:09:26.074 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-04-25T20:09:26.074 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-04-25T20:09:26.074 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_throttle_max_multiple": "0.000000", 2024-04-25T20:09:26.075 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_write_header_frequency": "0", 2024-04-25T20:09:26.075 INFO:teuthology.orchestra.run.smithi039.stdout: "journal_zero_on_create": "false", 2024-04-25T20:09:26.075 INFO:teuthology.orchestra.run.smithi039.stdout: "journaler_prefetch_periods": "10", 2024-04-25T20:09:26.075 INFO:teuthology.orchestra.run.smithi039.stdout: "journaler_prezero_periods": "5", 2024-04-25T20:09:26.075 INFO:teuthology.orchestra.run.smithi039.stdout: "journaler_write_head_interval": "15", 2024-04-25T20:09:26.075 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-04-25T20:09:26.075 INFO:teuthology.orchestra.run.smithi039.stdout: "keyfile": "", 2024-04-25T20:09:26.075 INFO:teuthology.orchestra.run.smithi039.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-04-25T20:09:26.075 INFO:teuthology.orchestra.run.smithi039.stdout: "kstore_backend": "rocksdb", 2024-04-25T20:09:26.075 INFO:teuthology.orchestra.run.smithi039.stdout: "kstore_default_stripe_size": "65536", 2024-04-25T20:09:26.075 INFO:teuthology.orchestra.run.smithi039.stdout: "kstore_fsck_on_mount": "false", 2024-04-25T20:09:26.075 INFO:teuthology.orchestra.run.smithi039.stdout: "kstore_fsck_on_mount_deep": "true", 2024-04-25T20:09:26.075 INFO:teuthology.orchestra.run.smithi039.stdout: "kstore_max_bytes": "67108864", 2024-04-25T20:09:26.075 INFO:teuthology.orchestra.run.smithi039.stdout: "kstore_max_ops": "512", 2024-04-25T20:09:26.075 INFO:teuthology.orchestra.run.smithi039.stdout: "kstore_nid_prealloc": "1024", 2024-04-25T20:09:26.076 INFO:teuthology.orchestra.run.smithi039.stdout: "kstore_onode_map_size": "1024", 2024-04-25T20:09:26.076 INFO:teuthology.orchestra.run.smithi039.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-04-25T20:09:26.076 INFO:teuthology.orchestra.run.smithi039.stdout: "kstore_sync_submit_transaction": "false", 2024-04-25T20:09:26.076 INFO:teuthology.orchestra.run.smithi039.stdout: "kstore_sync_transaction": "false", 2024-04-25T20:09:26.076 INFO:teuthology.orchestra.run.smithi039.stdout: "leveldb_block_size": "65536", 2024-04-25T20:09:26.076 INFO:teuthology.orchestra.run.smithi039.stdout: "leveldb_bloom_size": "0", 2024-04-25T20:09:26.076 INFO:teuthology.orchestra.run.smithi039.stdout: "leveldb_cache_size": "536870912", 2024-04-25T20:09:26.076 INFO:teuthology.orchestra.run.smithi039.stdout: "leveldb_compact_on_mount": "false", 2024-04-25T20:09:26.076 INFO:teuthology.orchestra.run.smithi039.stdout: "leveldb_compression": "false", 2024-04-25T20:09:26.076 INFO:teuthology.orchestra.run.smithi039.stdout: "leveldb_log": "", 2024-04-25T20:09:26.076 INFO:teuthology.orchestra.run.smithi039.stdout: "leveldb_log_to_ceph_log": "true", 2024-04-25T20:09:26.076 INFO:teuthology.orchestra.run.smithi039.stdout: "leveldb_max_open_files": "0", 2024-04-25T20:09:26.076 INFO:teuthology.orchestra.run.smithi039.stdout: "leveldb_paranoid": "false", 2024-04-25T20:09:26.076 INFO:teuthology.orchestra.run.smithi039.stdout: "leveldb_write_buffer_size": "33554432", 2024-04-25T20:09:26.077 INFO:teuthology.orchestra.run.smithi039.stdout: "librados_thread_count": "2", 2024-04-25T20:09:26.077 INFO:teuthology.orchestra.run.smithi039.stdout: "lockdep": "false", 2024-04-25T20:09:26.077 INFO:teuthology.orchestra.run.smithi039.stdout: "lockdep_force_backtrace": "false", 2024-04-25T20:09:26.077 INFO:teuthology.orchestra.run.smithi039.stdout: "log_coarse_timestamps": "true", 2024-04-25T20:09:26.077 INFO:teuthology.orchestra.run.smithi039.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-04-25T20:09:26.077 INFO:teuthology.orchestra.run.smithi039.stdout: "log_flush_on_exit": "false", 2024-04-25T20:09:26.077 INFO:teuthology.orchestra.run.smithi039.stdout: "log_graylog_host": "127.0.0.1", 2024-04-25T20:09:26.077 INFO:teuthology.orchestra.run.smithi039.stdout: "log_graylog_port": "12201", 2024-04-25T20:09:26.077 INFO:teuthology.orchestra.run.smithi039.stdout: "log_max_new": "1000", 2024-04-25T20:09:26.077 INFO:teuthology.orchestra.run.smithi039.stdout: "log_max_recent": "10000", 2024-04-25T20:09:26.077 INFO:teuthology.orchestra.run.smithi039.stdout: "log_stderr_prefix": "", 2024-04-25T20:09:26.077 INFO:teuthology.orchestra.run.smithi039.stdout: "log_stop_at_utilization": "0.970000", 2024-04-25T20:09:26.077 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": "true", 2024-04-25T20:09:26.077 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_graylog": "false", 2024-04-25T20:09:26.077 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_journald": "false", 2024-04-25T20:09:26.078 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_stderr": "false", 2024-04-25T20:09:26.078 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_syslog": "false", 2024-04-25T20:09:26.078 INFO:teuthology.orchestra.run.smithi039.stdout: "max_rotating_auth_attempts": "10", 2024-04-25T20:09:26.078 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-04-25T20:09:26.078 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_action_on_write_error": "1", 2024-04-25T20:09:26.078 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_alternate_name_max": "8192", 2024-04-25T20:09:26.078 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_asio_thread_count": "2", 2024-04-25T20:09:26.078 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_export_pin": "true", 2024-04-25T20:09:26.078 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_fragment_dirs": "true", 2024-04-25T20:09:26.078 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-04-25T20:09:26.078 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_fragment_interval": "5", 2024-04-25T20:09:26.078 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_fragment_size_max": "100000", 2024-04-25T20:09:26.078 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_idle_threshold": "0.000000", 2024-04-25T20:09:26.079 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_interval": "10", 2024-04-25T20:09:26.079 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_max": "-1", 2024-04-25T20:09:26.079 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_max_until": "-1", 2024-04-25T20:09:26.079 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_merge_size": "50", 2024-04-25T20:09:26.079 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_midchunk": "0.300000", 2024-04-25T20:09:26.079 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_min_rebalance": "0.100000", 2024-04-25T20:09:26.079 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_min_start": "0.200000", 2024-04-25T20:09:26.079 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_minchunk": "0.001000", 2024-04-25T20:09:26.079 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_mode": "0", 2024-04-25T20:09:26.079 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_need_max": "1.200000", 2024-04-25T20:09:26.079 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_need_min": "0.800000", 2024-04-25T20:09:26.079 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-04-25T20:09:26.079 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_sample_interval": "3.000000", 2024-04-25T20:09:26.079 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_split_bits": "3", 2024-04-25T20:09:26.079 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_split_rd": "25000.000000", 2024-04-25T20:09:26.080 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_split_size": "10000", 2024-04-25T20:09:26.080 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_split_wr": "10000.000000", 2024-04-25T20:09:26.080 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_target_decay": "10.000000", 2024-04-25T20:09:26.080 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-04-25T20:09:26.080 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_beacon_grace": "15.000000", 2024-04-25T20:09:26.080 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_beacon_interval": "4.000000", 2024-04-25T20:09:26.080 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_beacon_mon_down_grace": "60", 2024-04-25T20:09:26.080 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache_memory_limit": "4294967296", 2024-04-25T20:09:26.080 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache_mid": "0.700000", 2024-04-25T20:09:26.080 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache_release_free_interval": "10", 2024-04-25T20:09:26.080 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache_reservation": "0.050000", 2024-04-25T20:09:26.080 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-04-25T20:09:26.081 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache_trim_interval": "1", 2024-04-25T20:09:26.081 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache_trim_threshold": "262144", 2024-04-25T20:09:26.081 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-04-25T20:09:26.081 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-04-25T20:09:26.081 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_client_delegate_inos_pct": "50", 2024-04-25T20:09:26.081 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_client_prealloc_inos": "1000", 2024-04-25T20:09:26.081 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-04-25T20:09:26.081 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_connect_bootstrapping": "false", 2024-04-25T20:09:26.081 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_damage_table_max_entries": "10000", 2024-04-25T20:09:26.081 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-04-25T20:09:26.081 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_debug_auth_pins": "false", 2024-04-25T20:09:26.081 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_debug_frag": "false", 2024-04-25T20:09:26.081 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_debug_scatterstat": "false", 2024-04-25T20:09:26.081 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_debug_subtrees": "false", 2024-04-25T20:09:26.082 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_decay_halflife": "5.000000", 2024-04-25T20:09:26.082 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_default_dir_hash": "2", 2024-04-25T20:09:26.082 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_defer_session_stale": "true", 2024-04-25T20:09:26.082 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_deny_all_reconnect": "false", 2024-04-25T20:09:26.082 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_dir_keys_per_op": "16384", 2024-04-25T20:09:26.082 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_dir_max_commit_size": "10", 2024-04-25T20:09:26.082 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_dir_max_entries": "0", 2024-04-25T20:09:26.082 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_dir_prefetch": "true", 2024-04-25T20:09:26.082 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_dirstat_min_interval": "1.000000", 2024-04-25T20:09:26.082 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_dump_cache_after_rejoin": "false", 2024-04-25T20:09:26.082 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_dump_cache_on_map": "false", 2024-04-25T20:09:26.082 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_dump_cache_threshold_file": "0", 2024-04-25T20:09:26.082 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-04-25T20:09:26.083 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_early_reply": "true", 2024-04-25T20:09:26.083 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_enable_op_tracker": "true", 2024-04-25T20:09:26.083 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_enforce_unique_name": "true", 2024-04-25T20:09:26.083 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_export_ephemeral_distributed": "true", 2024-04-25T20:09:26.083 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-04-25T20:09:26.083 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_export_ephemeral_random": "true", 2024-04-25T20:09:26.083 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-04-25T20:09:26.083 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-04-25T20:09:26.083 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_forward_all_requests_to_auth": "false", 2024-04-25T20:09:26.083 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-04-25T20:09:26.083 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-04-25T20:09:26.083 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-04-25T20:09:26.083 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-04-25T20:09:26.083 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_health_cache_threshold": "1.500000", 2024-04-25T20:09:26.083 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_health_summarize_threshold": "10", 2024-04-25T20:09:26.084 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_heartbeat_grace": "15.000000", 2024-04-25T20:09:26.084 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_heartbeat_reset_grace": "1000", 2024-04-25T20:09:26.084 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_inject_health_dummy": "false", 2024-04-25T20:09:26.084 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-04-25T20:09:26.084 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_inject_migrator_session_race": "false", 2024-04-25T20:09:26.084 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-04-25T20:09:26.084 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-04-25T20:09:26.084 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-04-25T20:09:26.084 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_join_fs": "", 2024-04-25T20:09:26.084 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_journal_format": "1", 2024-04-25T20:09:26.084 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_kill_create_at": "0", 2024-04-25T20:09:26.084 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_kill_export_at": "0", 2024-04-25T20:09:26.085 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_kill_import_at": "0", 2024-04-25T20:09:26.085 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_kill_journal_at": "0", 2024-04-25T20:09:26.085 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_kill_journal_expire_at": "0", 2024-04-25T20:09:26.085 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_kill_journal_replay_at": "0", 2024-04-25T20:09:26.085 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_kill_link_at": "0", 2024-04-25T20:09:26.085 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_kill_mdstable_at": "0", 2024-04-25T20:09:26.085 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_kill_openc_at": "0", 2024-04-25T20:09:26.085 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_kill_rename_at": "0", 2024-04-25T20:09:26.085 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-04-25T20:09:26.085 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log_events_per_segment": "1024", 2024-04-25T20:09:26.085 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log_max_events": "-1", 2024-04-25T20:09:26.085 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log_max_segments": "128", 2024-04-25T20:09:26.086 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log_pause": "false", 2024-04-25T20:09:26.086 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log_segment_size": "0", 2024-04-25T20:09:26.086 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log_skip_corrupt_events": "false", 2024-04-25T20:09:26.086 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log_warn_factor": "2.000000", 2024-04-25T20:09:26.086 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_max_caps_per_client": "1048576", 2024-04-25T20:09:26.086 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_max_completed_flushes": "100000", 2024-04-25T20:09:26.086 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_max_completed_requests": "100000", 2024-04-25T20:09:26.086 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_max_export_size": "20971520", 2024-04-25T20:09:26.086 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_max_file_recover": "32", 2024-04-25T20:09:26.086 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_max_purge_files": "64", 2024-04-25T20:09:26.086 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_max_purge_ops": "8192", 2024-04-25T20:09:26.086 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-04-25T20:09:26.086 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-04-25T20:09:26.086 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_max_snaps_per_dir": "100", 2024-04-25T20:09:26.087 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_metrics_update_interval": "2", 2024-04-25T20:09:26.087 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_min_caps_per_client": "100", 2024-04-25T20:09:26.087 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_min_caps_working_set": "10000", 2024-04-25T20:09:26.087 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-04-25T20:09:26.087 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_numa_node": "-1", 2024-04-25T20:09:26.087 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-04-25T20:09:26.087 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_op_complaint_time": "30.000000", 2024-04-25T20:09:26.087 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_op_history_duration": "600", 2024-04-25T20:09:26.087 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_op_history_size": "20", 2024-04-25T20:09:26.087 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_op_history_slow_op_size": "20", 2024-04-25T20:09:26.087 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_op_history_slow_op_threshold": "10", 2024-04-25T20:09:26.087 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_op_log_threshold": "5", 2024-04-25T20:09:26.087 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_ping_grace": "15", 2024-04-25T20:09:26.087 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_ping_interval": "5", 2024-04-25T20:09:26.089 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-04-25T20:09:26.089 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-04-25T20:09:26.090 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_recall_max_caps": "30000", 2024-04-25T20:09:26.090 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-04-25T20:09:26.090 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_recall_max_decay_threshold": "131072", 2024-04-25T20:09:26.090 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-04-25T20:09:26.090 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_recall_warning_threshold": "262144", 2024-04-25T20:09:26.090 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_reconnect_timeout": "45.000000", 2024-04-25T20:09:26.090 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_replay_interval": "1.000000", 2024-04-25T20:09:26.090 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-04-25T20:09:26.090 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-04-25T20:09:26.090 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_root_ino_gid": "0", 2024-04-25T20:09:26.090 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_root_ino_uid": "0", 2024-04-25T20:09:26.090 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-04-25T20:09:26.090 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_session_blocklist_on_evict": "true", 2024-04-25T20:09:26.090 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_session_blocklist_on_timeout": "true", 2024-04-25T20:09:26.091 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-04-25T20:09:26.091 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-04-25T20:09:26.091 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-04-25T20:09:26.091 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-04-25T20:09:26.091 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-04-25T20:09:26.091 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_session_metadata_threshold": "16777216", 2024-04-25T20:09:26.091 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-04-25T20:09:26.091 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_shutdown_check": "0", 2024-04-25T20:09:26.091 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_skip_ino": "0", 2024-04-25T20:09:26.091 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_sleep_rank_change": "0.000000", 2024-04-25T20:09:26.091 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_snap_max_uid": "4294967294", 2024-04-25T20:09:26.091 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_snap_min_uid": "0", 2024-04-25T20:09:26.091 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_snap_rstat": "false", 2024-04-25T20:09:26.091 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_standby_replay_damaged": "false", 2024-04-25T20:09:26.091 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_symlink_recovery": "true", 2024-04-25T20:09:26.092 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_task_status_update_interval": "2.000000", 2024-04-25T20:09:26.092 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_thrash_exports": "0", 2024-04-25T20:09:26.092 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_thrash_fragments": "0", 2024-04-25T20:09:26.092 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_tick_interval": "5.000000", 2024-04-25T20:09:26.092 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_valgrind_exit": "false", 2024-04-25T20:09:26.092 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_verify_backtrace": "1", 2024-04-25T20:09:26.092 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_verify_scatter": "false", 2024-04-25T20:09:26.092 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_wipe_ino_prealloc": "false", 2024-04-25T20:09:26.092 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_wipe_sessions": "false", 2024-04-25T20:09:26.092 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool_debug": "false", 2024-04-25T20:09:26.092 INFO:teuthology.orchestra.run.smithi039.stdout: "memstore_debug_omit_block_device_write": "false", 2024-04-25T20:09:26.092 INFO:teuthology.orchestra.run.smithi039.stdout: "memstore_device_bytes": "1073741824", 2024-04-25T20:09:26.092 INFO:teuthology.orchestra.run.smithi039.stdout: "memstore_page_set": "false", 2024-04-25T20:09:26.092 INFO:teuthology.orchestra.run.smithi039.stdout: "memstore_page_size": "65536", 2024-04-25T20:09:26.093 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_client_bytes": "134217728", 2024-04-25T20:09:26.093 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_client_messages": "512", 2024-04-25T20:09:26.093 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-04-25T20:09:26.093 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_connect_retry_interval": "1.000000", 2024-04-25T20:09:26.093 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-04-25T20:09:26.093 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-04-25T20:09:26.093 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_disabled_modules": "", 2024-04-25T20:09:26.093 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-04-25T20:09:26.093 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_max_pg_num_change": "128", 2024-04-25T20:09:26.093 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_mds_bytes": "134217728", 2024-04-25T20:09:26.093 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_mds_messages": "128", 2024-04-25T20:09:26.093 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-04-25T20:09:26.093 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_mon_bytes": "134217728", 2024-04-25T20:09:26.093 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_mon_messages": "128", 2024-04-25T20:09:26.093 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_osd_bytes": "536870912", 2024-04-25T20:09:26.094 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_osd_messages": "8192", 2024-04-25T20:09:26.094 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_pool": "true", 2024-04-25T20:09:26.094 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_service_beacon_grace": "60.000000", 2024-04-25T20:09:26.094 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_standby_modules": "true", 2024-04-25T20:09:26.094 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_stats_period": "5", 2024-04-25T20:09:26.094 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_stats_threshold": "5", 2024-04-25T20:09:26.094 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_tick_period": "2", 2024-04-25T20:09:26.094 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-04-25T20:09:26.094 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_accept_timeout_factor": "2.000000", 2024-04-25T20:09:26.094 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_allow_pool_delete": "true", 2024-04-25T20:09:26.094 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_allow_pool_size_one": "true", 2024-04-25T20:09:26.094 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_auth_validate_all_caps": "true", 2024-04-25T20:09:26.094 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-04-25T20:09:26.095 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-04-25T20:09:26.095 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_client_bytes": "104857600", 2024-04-25T20:09:26.095 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_client_directed_command_retry": "5", 2024-04-25T20:09:26.095 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_client_hunt_interval": "3.000000", 2024-04-25T20:09:26.095 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-04-25T20:09:26.095 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-04-25T20:09:26.095 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-04-25T20:09:26.095 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_client_hunt_parallel": "3", 2024-04-25T20:09:26.095 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_client_log_interval": "1.000000", 2024-04-25T20:09:26.095 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-04-25T20:09:26.095 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_client_ping_interval": "10.000000", 2024-04-25T20:09:26.095 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_client_ping_timeout": "30.000000", 2024-04-25T20:09:26.095 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_clock_drift_allowed": "1.000000", 2024-04-25T20:09:26.095 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-04-25T20:09:26.095 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-04-25T20:09:26.096 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_cluster_log_file_level": "debug", 2024-04-25T20:09:26.096 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_cluster_log_to_file": "true", 2024-04-25T20:09:26.096 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_cluster_log_to_graylog": "false", 2024-04-25T20:09:26.096 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-04-25T20:09:26.096 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-04-25T20:09:26.096 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_cluster_log_to_journald": "false", 2024-04-25T20:09:26.096 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_cluster_log_to_stderr": "false", 2024-04-25T20:09:26.096 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-04-25T20:09:26.096 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-04-25T20:09:26.096 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-04-25T20:09:26.096 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_compact_on_bootstrap": "false", 2024-04-25T20:09:26.096 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_compact_on_start": "false", 2024-04-25T20:09:26.096 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_compact_on_trim": "true", 2024-04-25T20:09:26.096 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_con_tracker_persist_interval": "10", 2024-04-25T20:09:26.097 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_con_tracker_score_halflife": "43200", 2024-04-25T20:09:26.097 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_config_key_max_entry_size": "65536", 2024-04-25T20:09:26.097 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_cpu_threads": "4", 2024-04-25T20:09:26.097 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_crush_min_required_version": "hammer", 2024-04-25T20:09:26.097 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_daemon_bytes": "419430400", 2024-04-25T20:09:26.097 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-04-25T20:09:26.097 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_data_avail_crit": "5", 2024-04-25T20:09:26.097 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_data_avail_warn": "5", 2024-04-25T20:09:26.097 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_data_size_warn": "16106127360", 2024-04-25T20:09:26.097 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_debug_block_osdmap_trim": "false", 2024-04-25T20:09:26.097 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-04-25T20:09:26.097 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_debug_dump_json": "false", 2024-04-25T20:09:26.097 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-04-25T20:09:26.097 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_debug_dump_transactions": "false", 2024-04-25T20:09:26.098 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_debug_extra_checks": "false", 2024-04-25T20:09:26.098 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-04-25T20:09:26.098 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-04-25T20:09:26.098 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_debug_no_require_quincy": "false", 2024-04-25T20:09:26.098 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_debug_no_require_reef": "false", 2024-04-25T20:09:26.098 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-04-25T20:09:26.098 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_delta_reset_interval": "10.000000", 2024-04-25T20:09:26.098 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_dns_srv_name": "ceph-mon", 2024-04-25T20:09:26.098 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_down_mkfs_grace": "120", 2024-04-25T20:09:26.098 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_election_default_strategy": "1", 2024-04-25T20:09:26.098 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_election_timeout": "5.000000", 2024-04-25T20:09:26.098 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-04-25T20:09:26.098 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_elector_ping_divisor": "2", 2024-04-25T20:09:26.099 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_elector_ping_timeout": "2.000000", 2024-04-25T20:09:26.099 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_enable_op_tracker": "true", 2024-04-25T20:09:26.099 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_fake_pool_delete": "false", 2024-04-25T20:09:26.099 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_force_quorum_join": "false", 2024-04-25T20:09:26.099 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_globalid_prealloc": "10000", 2024-04-25T20:09:26.099 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_health_detail_to_clog": "false", 2024-04-25T20:09:26.099 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_health_log_update_period": "5", 2024-04-25T20:09:26.099 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_health_max_detail": "50", 2024-04-25T20:09:26.099 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_health_to_clog": "true", 2024-04-25T20:09:26.099 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_health_to_clog_interval": "600", 2024-04-25T20:09:26.099 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-04-25T20:09:26.099 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_host": "172.21.15.39,[v2:172.21.15.39:3301,v1:172.21.15.39:6790],172.21.15.135", 2024-04-25T20:09:26.099 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_host_override": "", 2024-04-25T20:09:26.099 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_initial_members": "", 2024-04-25T20:09:26.099 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-04-25T20:09:26.100 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-04-25T20:09:26.100 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-04-25T20:09:26.100 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-04-25T20:09:26.100 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_keyvaluedb": "rocksdb", 2024-04-25T20:09:26.100 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_lease": "5.000000", 2024-04-25T20:09:26.100 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-04-25T20:09:26.100 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-04-25T20:09:26.100 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_log_full_interval": "50", 2024-04-25T20:09:26.100 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_log_max": "10000", 2024-04-25T20:09:26.100 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_log_max_summary": "50", 2024-04-25T20:09:26.100 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_max_log_entries_per_event": "4096", 2024-04-25T20:09:26.100 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_max_log_epochs": "500", 2024-04-25T20:09:26.100 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_max_mdsmap_epochs": "500", 2024-04-25T20:09:26.101 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_max_mgrmap_epochs": "500", 2024-04-25T20:09:26.101 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_max_osd": "10000", 2024-04-25T20:09:26.101 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_max_pg_per_osd": "10000", 2024-04-25T20:09:26.101 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_max_pool_pg_num": "65536", 2024-04-25T20:09:26.101 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-04-25T20:09:26.101 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-04-25T20:09:26.101 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_mds_force_trim_to": "0", 2024-04-25T20:09:26.101 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_mds_skip_sanity": "false", 2024-04-25T20:09:26.101 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_memory_autotune": "true", 2024-04-25T20:09:26.101 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_memory_target": "2147483648", 2024-04-25T20:09:26.102 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_mgr_beacon_grace": "30", 2024-04-25T20:09:26.102 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-04-25T20:09:26.102 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_mgr_digest_period": "5", 2024-04-25T20:09:26.102 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_mgr_inactive_grace": "60", 2024-04-25T20:09:26.102 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_mgr_mkfs_grace": "240", 2024-04-25T20:09:26.102 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-04-25T20:09:26.102 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_min_osdmap_epochs": "500", 2024-04-25T20:09:26.102 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_op_complaint_time": "30", 2024-04-25T20:09:26.102 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_op_history_duration": "600", 2024-04-25T20:09:26.102 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_op_history_size": "20", 2024-04-25T20:09:26.102 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_op_history_slow_op_size": "20", 2024-04-25T20:09:26.102 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_op_history_slow_op_threshold": "10", 2024-04-25T20:09:26.102 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_op_log_threshold": "5", 2024-04-25T20:09:26.103 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-04-25T20:09:26.103 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-04-25T20:09:26.103 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-04-25T20:09:26.103 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_auto_mark_in": "false", 2024-04-25T20:09:26.103 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_auto_mark_new_in": "true", 2024-04-25T20:09:26.103 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-04-25T20:09:26.103 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-04-25T20:09:26.103 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_cache_size": "500", 2024-04-25T20:09:26.103 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_cache_size_min": "134217728", 2024-04-25T20:09:26.103 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_crush_smoke_test": "true", 2024-04-25T20:09:26.103 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_destroyed_out_interval": "600", 2024-04-25T20:09:26.103 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_down_out_interval": "600", 2024-04-25T20:09:26.103 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-04-25T20:09:26.103 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-04-25T20:09:26.103 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_force_trim_to": "0", 2024-04-25T20:09:26.104 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_full_ratio": "0.950000", 2024-04-25T20:09:26.104 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-04-25T20:09:26.104 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_laggy_halflife": "3600", 2024-04-25T20:09:26.104 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_laggy_max_interval": "300", 2024-04-25T20:09:26.104 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_laggy_weight": "0.300000", 2024-04-25T20:09:26.104 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-04-25T20:09:26.104 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_max_creating_pgs": "1024", 2024-04-25T20:09:26.104 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_max_initial_pgs": "1024", 2024-04-25T20:09:26.104 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_min_down_reporters": "2", 2024-04-25T20:09:26.104 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-04-25T20:09:26.104 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-04-25T20:09:26.104 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-04-25T20:09:26.104 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_prime_pg_temp": "true", 2024-04-25T20:09:26.104 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-04-25T20:09:26.105 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-04-25T20:09:26.105 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_report_timeout": "900", 2024-04-25T20:09:26.105 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-04-25T20:09:26.105 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-04-25T20:09:26.105 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_warn_num_repaired": "10", 2024-04-25T20:09:26.105 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osd_warn_op_age": "32.000000", 2024-04-25T20:09:26.105 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-04-25T20:09:26.105 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osdmap_full_prune_interval": "10", 2024-04-25T20:09:26.105 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osdmap_full_prune_min": "10000", 2024-04-25T20:09:26.105 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-04-25T20:09:26.105 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-04-25T20:09:26.105 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_pg_stuck_threshold": "60", 2024-04-25T20:09:26.105 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-04-25T20:09:26.105 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_pg_warn_min_objects": "10000", 2024-04-25T20:09:26.106 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_pg_warn_min_per_osd": "0", 2024-04-25T20:09:26.106 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-04-25T20:09:26.106 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_pool_quota_crit_threshold": "0", 2024-04-25T20:09:26.106 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_pool_quota_warn_threshold": "0", 2024-04-25T20:09:26.106 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_probe_timeout": "2.000000", 2024-04-25T20:09:26.106 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_reweight_max_change": "0.050000", 2024-04-25T20:09:26.106 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_reweight_max_osds": "4", 2024-04-25T20:09:26.106 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-04-25T20:09:26.106 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-04-25T20:09:26.106 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-04-25T20:09:26.106 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-04-25T20:09:26.106 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-04-25T20:09:26.106 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_scrub_interval": "86400", 2024-04-25T20:09:26.106 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_scrub_max_keys": "100", 2024-04-25T20:09:26.107 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_scrub_timeout": "300", 2024-04-25T20:09:26.107 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_session_timeout": "300", 2024-04-25T20:09:26.107 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_smart_report_timeout": "5", 2024-04-25T20:09:26.107 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_stat_smooth_intervals": "6", 2024-04-25T20:09:26.107 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-04-25T20:09:26.107 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_stretch_pool_min_size": "2", 2024-04-25T20:09:26.107 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_stretch_pool_size": "4", 2024-04-25T20:09:26.107 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-04-25T20:09:26.107 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_subscribe_interval": "86400.000000", 2024-04-25T20:09:26.107 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_sync_debug": "false", 2024-04-25T20:09:26.107 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_sync_max_payload_keys": "2000", 2024-04-25T20:09:26.107 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_sync_max_payload_size": "1048576", 2024-04-25T20:09:26.107 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_sync_provider_kill_at": "0", 2024-04-25T20:09:26.107 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_sync_requester_kill_at": "0", 2024-04-25T20:09:26.108 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_sync_timeout": "60.000000", 2024-04-25T20:09:26.108 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_target_pg_per_osd": "100", 2024-04-25T20:09:26.108 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_tick_interval": "5", 2024-04-25T20:09:26.108 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_timecheck_interval": "300.000000", 2024-04-25T20:09:26.108 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-04-25T20:09:26.108 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_use_min_delay_socket": "false", 2024-04-25T20:09:26.108 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_older_version_delay": "604800", 2024-04-25T20:09:26.108 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-04-25T20:09:26.108 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-04-25T20:09:26.108 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-04-25T20:09:26.108 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_filestore_osds": "false", 2024-04-25T20:09:26.108 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-04-25T20:09:26.108 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-04-25T20:09:26.108 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-04-25T20:09:26.109 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_misplaced": "false", 2024-04-25T20:09:26.109 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-04-25T20:09:26.109 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_older_version": "true", 2024-04-25T20:09:26.109 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-04-25T20:09:26.109 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_pool_no_app": "false", 2024-04-25T20:09:26.109 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-04-25T20:09:26.109 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-04-25T20:09:26.109 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-04-25T20:09:26.109 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-04-25T20:09:26.109 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_on_too_few_osds": "false", 2024-04-25T20:09:26.109 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-04-25T20:09:26.109 INFO:teuthology.orchestra.run.smithi039.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-04-25T20:09:26.109 INFO:teuthology.orchestra.run.smithi039.stdout: "monmap": "", 2024-04-25T20:09:26.109 INFO:teuthology.orchestra.run.smithi039.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-04-25T20:09:26.109 INFO:teuthology.orchestra.run.smithi039.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-04-25T20:09:26.110 INFO:teuthology.orchestra.run.smithi039.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-04-25T20:09:26.110 INFO:teuthology.orchestra.run.smithi039.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-04-25T20:09:26.110 INFO:teuthology.orchestra.run.smithi039.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-04-25T20:09:26.110 INFO:teuthology.orchestra.run.smithi039.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-04-25T20:09:26.110 INFO:teuthology.orchestra.run.smithi039.stdout: "motr_tracing_enabled": "false", 2024-04-25T20:09:26.110 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_op_threads": "3", 2024-04-25T20:09:26.110 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_buffer_size": "131072", 2024-04-25T20:09:26.110 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_cm": "false", 2024-04-25T20:09:26.110 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_device_name": "", 2024-04-25T20:09:26.110 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_dscp": "96", 2024-04-25T20:09:26.110 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-04-25T20:09:26.110 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_gid_idx": "0", 2024-04-25T20:09:26.110 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_local_gid": "", 2024-04-25T20:09:26.110 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_polling_us": "1000", 2024-04-25T20:09:26.111 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_port_num": "1", 2024-04-25T20:09:26.111 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-04-25T20:09:26.111 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-04-25T20:09:26.111 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_roce_ver": "1", 2024-04-25T20:09:26.111 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_send_buffers": "1024", 2024-04-25T20:09:26.111 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_sl": "3", 2024-04-25T20:09:26.111 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_support_srq": "true", 2024-04-25T20:09:26.111 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_rdma_type": "ib", 2024-04-25T20:09:26.111 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_async_reap_threshold": "5", 2024-04-25T20:09:26.111 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_bind_before_connect": "false", 2024-04-25T20:09:26.111 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_bind_ipv4": "true", 2024-04-25T20:09:26.111 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_bind_ipv6": "false", 2024-04-25T20:09:26.111 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_bind_msgr1": "true", 2024-04-25T20:09:26.111 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_bind_msgr2": "true", 2024-04-25T20:09:26.112 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_bind_port_max": "7568", 2024-04-25T20:09:26.112 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_bind_port_min": "6800", 2024-04-25T20:09:26.112 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_bind_prefer_ipv4": "false", 2024-04-25T20:09:26.112 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_bind_retry_count": "3", 2024-04-25T20:09:26.112 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_bind_retry_delay": "5", 2024-04-25T20:09:26.112 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_blackhole_client": "false", 2024-04-25T20:09:26.112 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_blackhole_mds": "false", 2024-04-25T20:09:26.112 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_blackhole_mgr": "false", 2024-04-25T20:09:26.112 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_blackhole_mon": "false", 2024-04-25T20:09:26.112 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_blackhole_osd": "false", 2024-04-25T20:09:26.112 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_client_mode": "crc secure", 2024-04-25T20:09:26.112 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_cluster_mode": "crc secure", 2024-04-25T20:09:26.112 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_cluster_type": "", 2024-04-25T20:09:26.112 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_compress_secure": "false", 2024-04-25T20:09:26.112 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_connection_idle_timeout": "900", 2024-04-25T20:09:26.113 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_connection_ready_timeout": "10", 2024-04-25T20:09:26.113 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_crc_data": "true", 2024-04-25T20:09:26.113 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_crc_header": "true", 2024-04-25T20:09:26.113 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_die_on_bad_msg": "false", 2024-04-25T20:09:26.113 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_die_on_bug": "true", 2024-04-25T20:09:26.113 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_die_on_old_message": "true", 2024-04-25T20:09:26.113 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_die_on_skipped_message": "false", 2024-04-25T20:09:26.113 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_die_on_unhandled_msg": "false", 2024-04-25T20:09:26.113 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-04-25T20:09:26.113 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_coremask": "0xF", 2024-04-25T20:09:26.113 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-04-25T20:09:26.113 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_devs_allowlist": "", 2024-04-25T20:09:26.113 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_enable_tso": "true", 2024-04-25T20:09:26.113 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-04-25T20:09:26.114 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-04-25T20:09:26.114 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_hugepages": "", 2024-04-25T20:09:26.114 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_hw_flow_control": "true", 2024-04-25T20:09:26.114 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-04-25T20:09:26.114 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_lro": "true", 2024-04-25T20:09:26.114 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_memory_channel": "4", 2024-04-25T20:09:26.114 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-04-25T20:09:26.114 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_pmd": "", 2024-04-25T20:09:26.114 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_port_id": "0", 2024-04-25T20:09:26.114 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-04-25T20:09:26.114 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dump_corrupt_message_level": "1", 2024-04-25T20:09:26.114 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_dump_on_send": "false", 2024-04-25T20:09:26.114 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_initial_backoff": "0.200000", 2024-04-25T20:09:26.114 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_inject_delay_max": "1.000000", 2024-04-25T20:09:26.115 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_inject_delay_probability": "0.000000", 2024-04-25T20:09:26.115 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_inject_delay_type": "", 2024-04-25T20:09:26.115 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_inject_internal_delays": "0.000000", 2024-04-25T20:09:26.115 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_inject_network_congestion": "0", 2024-04-25T20:09:26.115 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_inject_socket_failures": "5000", 2024-04-25T20:09:26.115 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_learn_addr_from_peer": "true", 2024-04-25T20:09:26.115 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_max_accept_failures": "4", 2024-04-25T20:09:26.115 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_max_backoff": "15.000000", 2024-04-25T20:09:26.115 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_mon_client_mode": "secure crc", 2024-04-25T20:09:26.115 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_mon_cluster_mode": "secure crc", 2024-04-25T20:09:26.115 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_mon_service_mode": "secure crc", 2024-04-25T20:09:26.115 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_osd_compress_min_size": "1024", 2024-04-25T20:09:26.115 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_osd_compress_mode": "none", 2024-04-25T20:09:26.115 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_osd_compression_algorithm": "snappy", 2024-04-25T20:09:26.116 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-04-25T20:09:26.116 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_pq_min_cost": "65536", 2024-04-25T20:09:26.116 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_public_type": "", 2024-04-25T20:09:26.116 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_service_mode": "crc secure", 2024-04-25T20:09:26.116 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_tcp_listen_backlog": "512", 2024-04-25T20:09:26.116 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_tcp_nodelay": "true", 2024-04-25T20:09:26.116 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-04-25T20:09:26.116 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_tcp_rcvbuf": "0", 2024-04-25T20:09:26.116 INFO:teuthology.orchestra.run.smithi039.stdout: "ms_type": "async+posix", 2024-04-25T20:09:26.116 INFO:teuthology.orchestra.run.smithi039.stdout: "no_config_file": "false", 2024-04-25T20:09:26.116 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter_completion_locks_per_session": "32", 2024-04-25T20:09:26.116 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter_debug_inject_relock_delay": "false", 2024-04-25T20:09:26.116 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter_inflight_op_bytes": "104857600", 2024-04-25T20:09:26.116 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter_inflight_ops": "1024", 2024-04-25T20:09:26.116 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter_inject_no_watch_ping": "false", 2024-04-25T20:09:26.117 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-04-25T20:09:26.117 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter_tick_interval": "5.000000", 2024-04-25T20:09:26.117 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter_timeout": "10.000000", 2024-04-25T20:09:26.117 INFO:teuthology.orchestra.run.smithi039.stdout: "objectstore_blackhole": "false", 2024-04-25T20:09:26.117 INFO:teuthology.orchestra.run.smithi039.stdout: "openssl_engine_opts": "", 2024-04-25T20:09:26.117 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_agent_delay_time": "5.000000", 2024-04-25T20:09:26.117 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_agent_hist_halflife": "1000", 2024-04-25T20:09:26.117 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_agent_max_low_ops": "2", 2024-04-25T20:09:26.117 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_agent_max_ops": "4", 2024-04-25T20:09:26.117 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-04-25T20:09:26.117 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_agent_quantize_effort": "0.100000", 2024-04-25T20:09:26.117 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_agent_slop": "0.020000", 2024-04-25T20:09:26.117 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-04-25T20:09:26.117 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_allow_recovery_below_min_size": "true", 2024-04-25T20:09:26.117 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_asio_thread_count": "2", 2024-04-25T20:09:26.118 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_async_recovery_min_cost": "100", 2024-04-25T20:09:26.118 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_auto_mark_unfound_lost": "false", 2024-04-25T20:09:26.118 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_backfill_retry_interval": "30.000000", 2024-04-25T20:09:26.118 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_backfill_scan_max": "512", 2024-04-25T20:09:26.118 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_backfill_scan_min": "64", 2024-04-25T20:09:26.118 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_backoff_on_degraded": "false", 2024-04-25T20:09:26.118 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_backoff_on_peering": "false", 2024-04-25T20:09:26.118 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_backoff_on_unfound": "true", 2024-04-25T20:09:26.118 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_beacon_report_interval": "300", 2024-04-25T20:09:26.118 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bench_duration": "30", 2024-04-25T20:09:26.118 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-04-25T20:09:26.119 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bench_max_block_size": "67108864", 2024-04-25T20:09:26.119 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bench_small_size_max_iops": "100", 2024-04-25T20:09:26.119 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_blkin_trace_all": "false", 2024-04-25T20:09:26.119 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_blocked_scrub_grace_period": "120", 2024-04-25T20:09:26.119 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-04-25T20:09:26.119 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-04-25T20:09:26.119 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-04-25T20:09:26.119 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_check_for_log_corruption": "false", 2024-04-25T20:09:26.119 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-04-25T20:09:26.119 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-04-25T20:09:26.119 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-04-25T20:09:26.120 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-04-25T20:09:26.120 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_class_update_on_start": "true", 2024-04-25T20:09:26.120 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_client_message_cap": "256", 2024-04-25T20:09:26.120 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_client_message_size_cap": "524288000", 2024-04-25T20:09:26.120 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_client_op_priority": "63", 2024-04-25T20:09:26.120 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_client_watch_timeout": "30", 2024-04-25T20:09:26.120 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_command_max_records": "256", 2024-04-25T20:09:26.120 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_compact_on_start": "false", 2024-04-25T20:09:26.120 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-04-25T20:09:26.120 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_crush_chooseleaf_type": "0", 2024-04-25T20:09:26.120 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_crush_initial_weight": "-1.000000", 2024-04-25T20:09:26.120 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_crush_update_on_start": "true", 2024-04-25T20:09:26.120 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_crush_update_weight_set": "true", 2024-04-25T20:09:26.120 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-04-25T20:09:26.121 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-04-25T20:09:26.121 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-04-25T20:09:26.121 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_drop_ping_duration": "0", 2024-04-25T20:09:26.121 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-04-25T20:09:26.121 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_feed_pullee": "-1", 2024-04-25T20:09:26.121 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-04-25T20:09:26.121 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-04-25T20:09:26.121 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-04-25T20:09:26.121 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_misdirected_ops": "false", 2024-04-25T20:09:26.121 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_no_acting_change": "false", 2024-04-25T20:09:26.121 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_no_purge_strays": "false", 2024-04-25T20:09:26.121 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_op_order": "false", 2024-04-25T20:09:26.121 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_pg_log_writeout": "false", 2024-04-25T20:09:26.121 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_pretend_recovery_active": "false", 2024-04-25T20:09:26.122 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-04-25T20:09:26.122 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-04-25T20:09:26.122 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_shutdown": "false", 2024-04-25T20:09:26.122 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-04-25T20:09:26.122 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-04-25T20:09:26.122 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_verify_cached_snaps": "false", 2024-04-25T20:09:26.122 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_verify_missing_on_start": "false", 2024-04-25T20:09:26.122 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_verify_snaps": "false", 2024-04-25T20:09:26.122 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-04-25T20:09:26.122 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-04-25T20:09:26.122 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_deep_scrub_keys": "1024", 2024-04-25T20:09:26.122 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-04-25T20:09:26.122 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-04-25T20:09:26.122 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-04-25T20:09:26.123 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_deep_scrub_stride": "524288", 2024-04-25T20:09:26.123 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-04-25T20:09:26.123 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_default_data_pool_replay_window": "5", 2024-04-25T20:09:26.123 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_default_notify_timeout": "30", 2024-04-25T20:09:26.123 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_delete_sleep": "0.000000", 2024-04-25T20:09:26.123 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-04-25T20:09:26.123 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-04-25T20:09:26.123 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-04-25T20:09:26.123 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_discard_disconnected_ops": "true", 2024-04-25T20:09:26.123 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_enable_op_tracker": "true", 2024-04-25T20:09:26.123 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-04-25T20:09:26.123 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_extblkdev_plugins": "vdo", 2024-04-25T20:09:26.123 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-04-25T20:09:26.124 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-04-25T20:09:26.124 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_fast_info": "true", 2024-04-25T20:09:26.124 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_fast_shutdown": "true", 2024-04-25T20:09:26.124 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-04-25T20:09:26.124 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_fast_shutdown_timeout": "15", 2024-04-25T20:09:26.124 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-04-25T20:09:26.124 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-04-25T20:09:26.124 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-04-25T20:09:26.124 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_function_tracing": "false", 2024-04-25T20:09:26.124 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_heartbeat_grace": "20", 2024-04-25T20:09:26.124 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_heartbeat_interval": "6", 2024-04-25T20:09:26.124 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-04-25T20:09:26.124 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_heartbeat_min_peers": "10", 2024-04-25T20:09:26.124 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_heartbeat_min_size": "2000", 2024-04-25T20:09:26.125 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_heartbeat_stale": "600", 2024-04-25T20:09:26.125 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-04-25T20:09:26.125 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_hit_set_max_size": "100000", 2024-04-25T20:09:26.125 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_hit_set_min_size": "1000", 2024-04-25T20:09:26.125 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-04-25T20:09:26.125 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-04-25T20:09:26.125 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-04-25T20:09:26.125 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-04-25T20:09:26.125 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-04-25T20:09:26.125 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_journal_flush_on_shutdown": "true", 2024-04-25T20:09:26.125 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_journal_size": "5120", 2024-04-25T20:09:26.125 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_kill_backfill_at": "0", 2024-04-25T20:09:26.125 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_loop_before_reset_tphandle": "64", 2024-04-25T20:09:26.125 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_map_cache_size": "50", 2024-04-25T20:09:26.125 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_map_dedup": "true", 2024-04-25T20:09:26.126 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_map_message_max": "40", 2024-04-25T20:09:26.126 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_map_message_max_bytes": "10485760", 2024-04-25T20:09:26.126 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_map_share_max_epochs": "40", 2024-04-25T20:09:26.126 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_attr_name_len": "100", 2024-04-25T20:09:26.126 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_attr_size": "0", 2024-04-25T20:09:26.126 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_backfills": "1", 2024-04-25T20:09:26.126 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_markdown_count": "5", 2024-04-25T20:09:26.126 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_markdown_period": "600", 2024-04-25T20:09:26.126 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_object_name_len": "2048", 2024-04-25T20:09:26.126 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_object_namespace_len": "256", 2024-04-25T20:09:26.126 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_object_size": "134217728", 2024-04-25T20:09:26.126 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-04-25T20:09:26.126 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_omap_entries_per_request": "1024", 2024-04-25T20:09:26.126 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_pg_blocked_by": "16", 2024-04-25T20:09:26.126 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_pg_log_entries": "10000", 2024-04-25T20:09:26.127 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-04-25T20:09:26.127 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_pgls": "1024", 2024-04-25T20:09:26.127 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_push_cost": "8388608", 2024-04-25T20:09:26.127 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_push_objects": "10", 2024-04-25T20:09:26.127 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_scrubs": "3", 2024-04-25T20:09:26.127 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-04-25T20:09:26.127 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_trimming_pgs": "2", 2024-04-25T20:09:26.127 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_write_op_reply_len": "64", 2024-04-25T20:09:26.127 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_max_write_size": "90", 2024-04-25T20:09:26.127 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-04-25T20:09:26.127 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-04-25T20:09:26.127 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-04-25T20:09:26.127 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-04-25T20:09:26.127 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-04-25T20:09:26.128 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-04-25T20:09:26.128 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-04-25T20:09:26.128 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_override_recovery_settings": "false", 2024-04-25T20:09:26.128 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_profile": "balanced", 2024-04-25T20:09:26.128 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-04-25T20:09:26.128 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-04-25T20:09:26.128 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-04-25T20:09:26.128 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-04-25T20:09:26.128 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-04-25T20:09:26.128 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-04-25T20:09:26.128 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-04-25T20:09:26.128 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-04-25T20:09:26.128 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-04-25T20:09:26.128 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-04-25T20:09:26.129 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mclock_skip_benchmark": "false", 2024-04-25T20:09:26.129 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_memory_base": "805306368", 2024-04-25T20:09:26.129 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_memory_cache_min": "134217728", 2024-04-25T20:09:26.129 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-04-25T20:09:26.129 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-04-25T20:09:26.129 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_memory_target": "4294967296", 2024-04-25T20:09:26.129 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_memory_target_autotune": "false", 2024-04-25T20:09:26.129 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-04-25T20:09:26.129 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_min_pg_log_entries": "250", 2024-04-25T20:09:26.129 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_min_recovery_priority": "0", 2024-04-25T20:09:26.129 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mon_heartbeat_interval": "30", 2024-04-25T20:09:26.129 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-04-25T20:09:26.129 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mon_report_interval": "5", 2024-04-25T20:09:26.129 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mon_report_max_in_flight": "2", 2024-04-25T20:09:26.129 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-04-25T20:09:26.130 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_num_cache_shards": "32", 2024-04-25T20:09:26.130 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_num_op_tracker_shard": "32", 2024-04-25T20:09:26.130 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_numa_auto_affinity": "true", 2024-04-25T20:09:26.130 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_numa_node": "-1", 2024-04-25T20:09:26.130 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_numa_prefer_iface": "true", 2024-04-25T20:09:26.130 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-04-25T20:09:26.130 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_objecter_finishers": "1", 2024-04-25T20:09:26.130 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_objectstore": "bluestore", 2024-04-25T20:09:26.130 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_objectstore_fuse": "false", 2024-04-25T20:09:26.130 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_objectstore_tracing": "false", 2024-04-25T20:09:26.130 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_complaint_time": "30.000000", 2024-04-25T20:09:26.130 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_history_duration": "600", 2024-04-25T20:09:26.130 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_history_size": "20", 2024-04-25T20:09:26.130 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_history_slow_op_size": "20", 2024-04-25T20:09:26.131 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-04-25T20:09:26.131 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_log_threshold": "5", 2024-04-25T20:09:26.131 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_num_shards": "0", 2024-04-25T20:09:26.131 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_num_shards_hdd": "5", 2024-04-25T20:09:26.131 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_num_shards_ssd": "8", 2024-04-25T20:09:26.131 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_num_threads_per_shard": "0", 2024-04-25T20:09:26.131 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-04-25T20:09:26.131 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-04-25T20:09:26.131 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-04-25T20:09:26.131 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_pq_min_cost": "65536", 2024-04-25T20:09:26.131 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_queue": "mclock_scheduler", 2024-04-25T20:09:26.131 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_queue_cut_off": "high", 2024-04-25T20:09:26.131 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_thread_suicide_timeout": "150", 2024-04-25T20:09:26.131 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_op_thread_timeout": "15", 2024-04-25T20:09:26.132 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_open_classes_on_start": "true", 2024-04-25T20:09:26.132 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_os_flags": "0", 2024-04-25T20:09:26.132 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_peering_op_priority": "255", 2024-04-25T20:09:26.132 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pg_delete_cost": "1048576", 2024-04-25T20:09:26.132 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pg_delete_priority": "5", 2024-04-25T20:09:26.132 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-04-25T20:09:26.132 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-04-25T20:09:26.132 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pg_log_dups_tracked": "3000", 2024-04-25T20:09:26.132 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pg_log_trim_max": "10000", 2024-04-25T20:09:26.132 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pg_log_trim_min": "100", 2024-04-25T20:09:26.132 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-04-25T20:09:26.132 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pg_object_context_cache_count": "64", 2024-04-25T20:09:26.132 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pg_stat_report_interval_max": "500", 2024-04-25T20:09:26.132 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-04-25T20:09:26.132 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-04-25T20:09:26.133 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-04-25T20:09:26.133 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-04-25T20:09:26.133 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-04-25T20:09:26.133 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-04-25T20:09:26.133 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_crimson": "false", 2024-04-25T20:09:26.133 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_crush_rule": "-1", 2024-04-25T20:09:26.133 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_ec_fast_read": "false", 2024-04-25T20:09:26.133 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-04-25T20:09:26.133 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_flag_bulk": "false", 2024-04-25T20:09:26.133 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-04-25T20:09:26.133 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_flag_nodelete": "false", 2024-04-25T20:09:26.133 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-04-25T20:09:26.133 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-04-25T20:09:26.133 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_flags": "0", 2024-04-25T20:09:26.134 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-04-25T20:09:26.134 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_min_size": "0", 2024-04-25T20:09:26.134 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-04-25T20:09:26.134 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_pg_num": "32", 2024-04-25T20:09:26.134 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_pgp_num": "0", 2024-04-25T20:09:26.134 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-04-25T20:09:26.134 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_size": "2", 2024-04-25T20:09:26.134 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_default_type": "replicated", 2024-04-25T20:09:26.134 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-04-25T20:09:26.134 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pool_use_gmt_hitset": "true", 2024-04-25T20:09:26.134 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_push_per_object_cost": "1000", 2024-04-25T20:09:26.134 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_read_ec_check_for_errors": "false", 2024-04-25T20:09:26.134 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recover_clone_overlap": "true", 2024-04-25T20:09:26.134 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recover_clone_overlap_limit": "10", 2024-04-25T20:09:26.134 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_cost": "20971520", 2024-04-25T20:09:26.135 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_delay_start": "0.000000", 2024-04-25T20:09:26.135 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_max_active": "0", 2024-04-25T20:09:26.135 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_max_active_hdd": "3", 2024-04-25T20:09:26.135 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_max_active_ssd": "10", 2024-04-25T20:09:26.135 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_max_chunk": "8388608", 2024-04-25T20:09:26.135 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-04-25T20:09:26.135 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_max_single_start": "1", 2024-04-25T20:09:26.135 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_op_priority": "3", 2024-04-25T20:09:26.135 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_op_warn_multiple": "16", 2024-04-25T20:09:26.135 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_priority": "5", 2024-04-25T20:09:26.135 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_retry_interval": "30.000000", 2024-04-25T20:09:26.135 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_sleep": "0.000000", 2024-04-25T20:09:26.135 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-04-25T20:09:26.136 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-04-25T20:09:26.136 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-04-25T20:09:26.136 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_repair_during_recovery": "false", 2024-04-25T20:09:26.136 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_requested_scrub_priority": "120", 2024-04-25T20:09:26.136 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-04-25T20:09:26.136 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_rollback_to_cluster_snap": "", 2024-04-25T20:09:26.136 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_auto_repair": "false", 2024-04-25T20:09:26.136 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-04-25T20:09:26.136 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-04-25T20:09:26.136 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_begin_hour": "0", 2024-04-25T20:09:26.136 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_begin_week_day": "0", 2024-04-25T20:09:26.137 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_chunk_max": "25", 2024-04-25T20:09:26.137 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_chunk_min": "5", 2024-04-25T20:09:26.137 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_cost": "52428800", 2024-04-25T20:09:26.137 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_during_recovery": "false", 2024-04-25T20:09:26.137 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_end_hour": "0", 2024-04-25T20:09:26.137 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_end_week_day": "0", 2024-04-25T20:09:26.137 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_event_cost": "4096", 2024-04-25T20:09:26.137 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-04-25T20:09:26.137 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-04-25T20:09:26.137 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_invalid_stats": "true", 2024-04-25T20:09:26.137 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_load_threshold": "0.500000", 2024-04-25T20:09:26.137 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_max_interval": "604800.000000", 2024-04-25T20:09:26.138 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_max_preemptions": "5", 2024-04-25T20:09:26.138 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_min_interval": "86400.000000", 2024-04-25T20:09:26.138 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_priority": "5", 2024-04-25T20:09:26.138 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_reservation_timeout": "5000", 2024-04-25T20:09:26.138 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_sleep": "0.000000", 2024-04-25T20:09:26.138 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_scrub_slow_reservation_response": "2200", 2024-04-25T20:09:26.138 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-04-25T20:09:26.138 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-04-25T20:09:26.138 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_shutdown_pgref_assert": "false", 2024-04-25T20:09:26.138 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_skip_data_digest": "false", 2024-04-25T20:09:26.138 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_smart_report_timeout": "5", 2024-04-25T20:09:26.138 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_snap_trim_cost": "1048576", 2024-04-25T20:09:26.138 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_snap_trim_priority": "5", 2024-04-25T20:09:26.138 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_snap_trim_sleep": "0.000000", 2024-04-25T20:09:26.139 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-04-25T20:09:26.139 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-04-25T20:09:26.139 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-04-25T20:09:26.139 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-04-25T20:09:26.139 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_stats_update_period_scrubbing": "15", 2024-04-25T20:09:26.139 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-04-25T20:09:26.139 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_target_transaction_size": "30", 2024-04-25T20:09:26.139 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-04-25T20:09:26.139 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-04-25T20:09:26.139 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-04-25T20:09:26.139 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-04-25T20:09:26.139 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-04-25T20:09:26.139 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-04-25T20:09:26.139 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-04-25T20:09:26.140 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_tier_default_cache_mode": "writeback", 2024-04-25T20:09:26.140 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-04-25T20:09:26.140 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-04-25T20:09:26.140 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_tracing": "false", 2024-04-25T20:09:26.140 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_use_stale_snap": "false", 2024-04-25T20:09:26.140 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T20:09:26.140 INFO:teuthology.orchestra.run.smithi039.stdout: "osdc_blkin_trace_all": "false", 2024-04-25T20:09:26.140 INFO:teuthology.orchestra.run.smithi039.stdout: "paxos_kill_at": "0", 2024-04-25T20:09:26.140 INFO:teuthology.orchestra.run.smithi039.stdout: "paxos_max_join_drift": "10", 2024-04-25T20:09:26.140 INFO:teuthology.orchestra.run.smithi039.stdout: "paxos_min": "500", 2024-04-25T20:09:26.140 INFO:teuthology.orchestra.run.smithi039.stdout: "paxos_min_wait": "0.050000", 2024-04-25T20:09:26.140 INFO:teuthology.orchestra.run.smithi039.stdout: "paxos_propose_interval": "1.000000", 2024-04-25T20:09:26.140 INFO:teuthology.orchestra.run.smithi039.stdout: "paxos_service_trim_max": "500", 2024-04-25T20:09:26.141 INFO:teuthology.orchestra.run.smithi039.stdout: "paxos_service_trim_max_multiplier": "20", 2024-04-25T20:09:26.141 INFO:teuthology.orchestra.run.smithi039.stdout: "paxos_service_trim_min": "250", 2024-04-25T20:09:26.141 INFO:teuthology.orchestra.run.smithi039.stdout: "paxos_stash_full_interval": "25", 2024-04-25T20:09:26.141 INFO:teuthology.orchestra.run.smithi039.stdout: "paxos_trim_max": "500", 2024-04-25T20:09:26.141 INFO:teuthology.orchestra.run.smithi039.stdout: "paxos_trim_min": "250", 2024-04-25T20:09:26.141 INFO:teuthology.orchestra.run.smithi039.stdout: "perf": "true", 2024-04-25T20:09:26.141 INFO:teuthology.orchestra.run.smithi039.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-04-25T20:09:26.141 INFO:teuthology.orchestra.run.smithi039.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-04-25T20:09:26.141 INFO:teuthology.orchestra.run.smithi039.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-04-25T20:09:26.141 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "-", 2024-04-25T20:09:26.141 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrv": "", 2024-04-25T20:09:26.141 INFO:teuthology.orchestra.run.smithi039.stdout: "public_bind_addr": "-", 2024-04-25T20:09:26.141 INFO:teuthology.orchestra.run.smithi039.stdout: "public_network": "", 2024-04-25T20:09:26.141 INFO:teuthology.orchestra.run.smithi039.stdout: "public_network_interface": "", 2024-04-25T20:09:26.142 INFO:teuthology.orchestra.run.smithi039.stdout: "qat_compressor_enabled": "false", 2024-04-25T20:09:26.142 INFO:teuthology.orchestra.run.smithi039.stdout: "qat_compressor_session_max_number": "256", 2024-04-25T20:09:26.142 INFO:teuthology.orchestra.run.smithi039.stdout: "rados_mon_op_timeout": "0", 2024-04-25T20:09:26.142 INFO:teuthology.orchestra.run.smithi039.stdout: "rados_osd_op_timeout": "0", 2024-04-25T20:09:26.142 INFO:teuthology.orchestra.run.smithi039.stdout: "rados_tracing": "false", 2024-04-25T20:09:26.142 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_atime_update_interval": "60", 2024-04-25T20:09:26.142 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-04-25T20:09:26.142 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_balance_parent_reads": "false", 2024-04-25T20:09:26.142 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_balance_snap_reads": "false", 2024-04-25T20:09:26.142 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_blkin_trace_all": "false", 2024-04-25T20:09:26.142 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_blocklist_expire_seconds": "0", 2024-04-25T20:09:26.142 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_blocklist_on_break_lock": "true", 2024-04-25T20:09:26.142 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_cache": "true", 2024-04-25T20:09:26.142 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_cache_block_writes_upfront": "false", 2024-04-25T20:09:26.143 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_cache_max_dirty": "25165824", 2024-04-25T20:09:26.143 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-04-25T20:09:26.143 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_cache_max_dirty_object": "0", 2024-04-25T20:09:26.143 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_cache_policy": "writearound", 2024-04-25T20:09:26.143 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_cache_size": "33554432", 2024-04-25T20:09:26.143 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_cache_target_dirty": "16777216", 2024-04-25T20:09:26.143 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-04-25T20:09:26.143 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_clone_copy_on_read": "false", 2024-04-25T20:09:26.143 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_compression_hint": "none", 2024-04-25T20:09:26.143 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_concurrent_management_ops": "10", 2024-04-25T20:09:26.143 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-04-25T20:09:26.143 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_default_clone_format": "auto", 2024-04-25T20:09:26.143 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_default_data_pool": "", 2024-04-25T20:09:26.143 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_default_features": "61", 2024-04-25T20:09:26.143 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_default_format": "2", 2024-04-25T20:09:26.144 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_default_map_options": "", 2024-04-25T20:09:26.144 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_default_order": "22", 2024-04-25T20:09:26.144 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_default_pool": "rbd", 2024-04-25T20:09:26.144 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-04-25T20:09:26.144 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_default_stripe_count": "0", 2024-04-25T20:09:26.144 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_default_stripe_unit": "0", 2024-04-25T20:09:26.144 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_disable_zero_copy_writes": "true", 2024-04-25T20:09:26.144 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_discard_granularity_bytes": "65536", 2024-04-25T20:09:26.144 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-04-25T20:09:26.144 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_enable_alloc_hint": "true", 2024-04-25T20:09:26.144 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-04-25T20:09:26.144 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_io_scheduler": "simple", 2024-04-25T20:09:26.144 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-04-25T20:09:26.144 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_journal_commit_age": "5.000000", 2024-04-25T20:09:26.144 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-04-25T20:09:26.145 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-04-25T20:09:26.145 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-04-25T20:09:26.145 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-04-25T20:09:26.145 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_journal_object_flush_interval": "0", 2024-04-25T20:09:26.145 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-04-25T20:09:26.145 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-04-25T20:09:26.145 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_journal_order": "24", 2024-04-25T20:09:26.145 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_journal_pool": "", 2024-04-25T20:09:26.145 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_journal_splay_width": "4", 2024-04-25T20:09:26.145 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_localize_parent_reads": "false", 2024-04-25T20:09:26.145 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_localize_snap_reads": "false", 2024-04-25T20:09:26.145 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-04-25T20:09:26.145 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-04-25T20:09:26.145 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-04-25T20:09:26.146 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-04-25T20:09:26.146 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-04-25T20:09:26.146 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-04-25T20:09:26.146 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_image_policy_type": "simple", 2024-04-25T20:09:26.146 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-04-25T20:09:26.146 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-04-25T20:09:26.146 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-04-25T20:09:26.146 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-04-25T20:09:26.146 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-04-25T20:09:26.146 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-04-25T20:09:26.146 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-04-25T20:09:26.146 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_memory_autotune": "true", 2024-04-25T20:09:26.146 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_memory_base": "805306368", 2024-04-25T20:09:26.146 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-04-25T20:09:26.147 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-04-25T20:09:26.147 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-04-25T20:09:26.147 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-04-25T20:09:26.147 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_memory_target": "4294967296", 2024-04-25T20:09:26.147 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-04-25T20:09:26.147 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-04-25T20:09:26.147 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-04-25T20:09:26.147 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirroring_delete_delay": "0", 2024-04-25T20:09:26.147 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-04-25T20:09:26.147 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirroring_replay_delay": "0", 2024-04-25T20:09:26.147 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-04-25T20:09:26.147 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-04-25T20:09:26.147 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_move_to_trash_on_remove": "false", 2024-04-25T20:09:26.147 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-04-25T20:09:26.147 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_mtime_update_interval": "60", 2024-04-25T20:09:26.148 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_non_blocking_aio": "true", 2024-04-25T20:09:26.148 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_op_thread_timeout": "60", 2024-04-25T20:09:26.148 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_op_threads": "1", 2024-04-25T20:09:26.148 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_parent_cache_enabled": "false", 2024-04-25T20:09:26.148 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_persistent_cache_mode": "disabled", 2024-04-25T20:09:26.148 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_persistent_cache_path": "/tmp", 2024-04-25T20:09:26.148 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_persistent_cache_size": "1073741824", 2024-04-25T20:09:26.148 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_plugins": "", 2024-04-25T20:09:26.148 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_bps_burst": "0", 2024-04-25T20:09:26.148 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-04-25T20:09:26.148 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_bps_limit": "0", 2024-04-25T20:09:26.148 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_exclude_ops": "0", 2024-04-25T20:09:26.148 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_iops_burst": "0", 2024-04-25T20:09:26.148 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-04-25T20:09:26.149 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_iops_limit": "0", 2024-04-25T20:09:26.149 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_read_bps_burst": "0", 2024-04-25T20:09:26.149 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-04-25T20:09:26.149 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_read_bps_limit": "0", 2024-04-25T20:09:26.149 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_read_iops_burst": "0", 2024-04-25T20:09:26.149 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-04-25T20:09:26.149 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_read_iops_limit": "0", 2024-04-25T20:09:26.149 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_schedule_tick_min": "50", 2024-04-25T20:09:26.149 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_write_bps_burst": "0", 2024-04-25T20:09:26.149 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-04-25T20:09:26.149 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_write_bps_limit": "0", 2024-04-25T20:09:26.149 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_write_iops_burst": "0", 2024-04-25T20:09:26.149 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-04-25T20:09:26.149 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_qos_write_iops_limit": "0", 2024-04-25T20:09:26.149 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_quiesce_notification_attempts": "10", 2024-04-25T20:09:26.150 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_read_from_replica_policy": "default", 2024-04-25T20:09:26.150 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-04-25T20:09:26.150 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_readahead_max_bytes": "524288", 2024-04-25T20:09:26.150 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_readahead_trigger_requests": "10", 2024-04-25T20:09:26.150 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_request_timed_out_seconds": "30", 2024-04-25T20:09:26.150 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_skip_partial_discard": "true", 2024-04-25T20:09:26.150 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-04-25T20:09:26.150 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_tracing": "false", 2024-04-25T20:09:26.150 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_validate_names": "true", 2024-04-25T20:09:26.150 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_validate_pool": "true", 2024-04-25T20:09:26.150 INFO:teuthology.orchestra.run.smithi039.stdout: "restapi_base_url": "", 2024-04-25T20:09:26.150 INFO:teuthology.orchestra.run.smithi039.stdout: "restapi_log_level": "", 2024-04-25T20:09:26.150 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_acl_grants_max_num": "100", 2024-04-25T20:09:26.150 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_admin_entry": "admin", 2024-04-25T20:09:26.151 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-04-25T20:09:26.151 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_backend_store": "rados", 2024-04-25T20:09:26.151 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_barbican_url": "", 2024-04-25T20:09:26.151 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_beast_enable_async": "true", 2024-04-25T20:09:26.151 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-04-25T20:09:26.151 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-04-25T20:09:26.151 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_bucket_index_max_aio": "128", 2024-04-25T20:09:26.151 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-04-25T20:09:26.151 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-04-25T20:09:26.151 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_bucket_quota_ttl": "600", 2024-04-25T20:09:26.151 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-04-25T20:09:26.151 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_cache_enabled": "true", 2024-04-25T20:09:26.151 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_cache_expiry_interval": "900", 2024-04-25T20:09:26.151 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_cache_lru_size": "10000", 2024-04-25T20:09:26.151 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_config_store": "rados", 2024-04-25T20:09:26.152 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_content_length_compat": "false", 2024-04-25T20:09:26.152 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_copy_obj_progress": "true", 2024-04-25T20:09:26.152 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-04-25T20:09:26.152 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_cors_rules_max_num": "100", 2024-04-25T20:09:26.152 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_cross_domain_policy": "", 2024-04-25T20:09:26.152 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_default_encryption_key": "", 2024-04-25T20:09:26.152 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_kmip_addr": "", 2024-04-25T20:09:26.152 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_kmip_ca_path": "", 2024-04-25T20:09:26.152 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_kmip_client_cert": "", 2024-04-25T20:09:26.152 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_kmip_client_key": "", 2024-04-25T20:09:26.152 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-04-25T20:09:26.152 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_kmip_password": "", 2024-04-25T20:09:26.152 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-04-25T20:09:26.153 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_kmip_username": "", 2024-04-25T20:09:26.153 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_require_ssl": "true", 2024-04-25T20:09:26.153 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-04-25T20:09:26.153 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-04-25T20:09:26.153 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-04-25T20:09:26.153 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-04-25T20:09:26.153 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-04-25T20:09:26.153 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-04-25T20:09:26.153 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-04-25T20:09:26.153 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-04-25T20:09:26.154 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-04-25T20:09:26.154 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-04-25T20:09:26.154 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-04-25T20:09:26.154 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-04-25T20:09:26.154 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-04-25T20:09:26.154 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-04-25T20:09:26.154 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_suppress_logs": "true", 2024-04-25T20:09:26.154 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_vault_addr": "", 2024-04-25T20:09:26.154 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_vault_auth": "token", 2024-04-25T20:09:26.154 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_vault_namespace": "", 2024-04-25T20:09:26.154 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_vault_prefix": "", 2024-04-25T20:09:26.154 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-04-25T20:09:26.154 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-04-25T20:09:26.154 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-04-25T20:09:26.155 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-04-25T20:09:26.155 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_vault_token_file": "", 2024-04-25T20:09:26.155 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-04-25T20:09:26.155 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_curl_buffersize": "524288", 2024-04-25T20:09:26.155 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_curl_low_speed_limit": "1024", 2024-04-25T20:09:26.155 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_curl_low_speed_time": "300", 2024-04-25T20:09:26.155 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_curl_tcp_keepalive": "0", 2024-04-25T20:09:26.155 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-04-25T20:09:26.155 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-04-25T20:09:26.155 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-04-25T20:09:26.155 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-04-25T20:09:26.155 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-04-25T20:09:26.155 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_d3n_l1_fadvise": "4", 2024-04-25T20:09:26.155 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-04-25T20:09:26.156 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-04-25T20:09:26.156 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-04-25T20:09:26.156 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-04-25T20:09:26.156 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_data_log_changes_size": "1000", 2024-04-25T20:09:26.156 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_data_log_num_shards": "128", 2024-04-25T20:09:26.156 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-04-25T20:09:26.156 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_data_log_window": "30", 2024-04-25T20:09:26.156 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_data_notify_interval_msec": "0", 2024-04-25T20:09:26.156 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_data_sync_poll_interval": "20", 2024-04-25T20:09:26.156 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_data_sync_spawn_window": "20", 2024-04-25T20:09:26.156 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-04-25T20:09:26.156 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-04-25T20:09:26.156 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_default_data_log_backing": "fifo", 2024-04-25T20:09:26.156 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-04-25T20:09:26.156 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_default_region_info_oid": "default.region", 2024-04-25T20:09:26.157 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-04-25T20:09:26.157 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-04-25T20:09:26.157 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_defer_to_bucket_acls": "", 2024-04-25T20:09:26.157 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-04-25T20:09:26.157 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-04-25T20:09:26.157 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-04-25T20:09:26.157 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-04-25T20:09:26.157 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-04-25T20:09:26.157 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-04-25T20:09:26.157 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-04-25T20:09:26.157 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-04-25T20:09:26.157 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dmclock_data_res": "500.000000", 2024-04-25T20:09:26.157 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-04-25T20:09:26.157 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-04-25T20:09:26.158 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-04-25T20:09:26.158 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-04-25T20:09:26.158 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dns_name": "", 2024-04-25T20:09:26.158 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dns_s3website_name": "", 2024-04-25T20:09:26.158 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_dynamic_resharding": "true", 2024-04-25T20:09:26.158 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-04-25T20:09:26.158 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_enable_gc_threads": "true", 2024-04-25T20:09:26.158 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_enable_lc_threads": "true", 2024-04-25T20:09:26.158 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_enable_ops_log": "false", 2024-04-25T20:09:26.158 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_enable_quota_threads": "true", 2024-04-25T20:09:26.158 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_enable_static_website": "false", 2024-04-25T20:09:26.158 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_enable_usage_log": "false", 2024-04-25T20:09:26.158 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_enforce_swift_acls": "true", 2024-04-25T20:09:26.159 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_exit_timeout_secs": "120", 2024-04-25T20:09:26.159 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_expose_bucket": "false", 2024-04-25T20:09:26.159 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_extended_http_attrs": "", 2024-04-25T20:09:26.159 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_filter": "none", 2024-04-25T20:09:26.159 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-04-25T20:09:26.159 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_frontends": "beast port=7480", 2024-04-25T20:09:26.159 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_gc_max_concurrent_io": "10", 2024-04-25T20:09:26.159 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_gc_max_deferred": "50", 2024-04-25T20:09:26.159 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-04-25T20:09:26.159 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_gc_max_objs": "32", 2024-04-25T20:09:26.159 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_gc_max_queue_size": "134213632", 2024-04-25T20:09:26.159 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_gc_max_trim_chunk": "16", 2024-04-25T20:09:26.159 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_gc_obj_min_wait": "7200", 2024-04-25T20:09:26.159 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_gc_processor_max_time": "3600", 2024-04-25T20:09:26.159 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_gc_processor_period": "3600", 2024-04-25T20:09:26.160 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-04-25T20:09:26.160 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_get_obj_window_size": "16777216", 2024-04-25T20:09:26.160 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_healthcheck_disabling_path": "", 2024-04-25T20:09:26.160 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_ignore_get_invalid_range": "false", 2024-04-25T20:09:26.160 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_init_timeout": "300", 2024-04-25T20:09:26.160 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-04-25T20:09:26.160 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-04-25T20:09:26.160 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-04-25T20:09:26.160 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-04-25T20:09:26.160 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_admin_domain": "", 2024-04-25T20:09:26.160 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_admin_password": "", 2024-04-25T20:09:26.160 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_admin_password_path": "", 2024-04-25T20:09:26.160 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_admin_project": "", 2024-04-25T20:09:26.160 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_admin_tenant": "", 2024-04-25T20:09:26.161 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_admin_token": "", 2024-04-25T20:09:26.161 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_admin_token_path": "", 2024-04-25T20:09:26.161 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_admin_user": "", 2024-04-25T20:09:26.161 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_api_version": "2", 2024-04-25T20:09:26.161 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_barbican_domain": "", 2024-04-25T20:09:26.161 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_barbican_password": "", 2024-04-25T20:09:26.161 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_barbican_project": "", 2024-04-25T20:09:26.161 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_barbican_tenant": "", 2024-04-25T20:09:26.161 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_barbican_user": "", 2024-04-25T20:09:26.161 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-04-25T20:09:26.161 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_implicit_tenants": "false", 2024-04-25T20:09:26.161 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-04-25T20:09:26.161 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_service_token_enabled": "false", 2024-04-25T20:09:26.161 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_token_cache_size": "10000", 2024-04-25T20:09:26.161 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_url": "", 2024-04-25T20:09:26.161 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_keystone_verify_ssl": "true", 2024-04-25T20:09:26.162 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_lc_debug_interval": "-1", 2024-04-25T20:09:26.162 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_lc_lock_max_time": "90", 2024-04-25T20:09:26.162 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_lc_max_objs": "32", 2024-04-25T20:09:26.162 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_lc_max_rules": "1000", 2024-04-25T20:09:26.162 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_lc_max_worker": "3", 2024-04-25T20:09:26.162 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_lc_max_wp_worker": "3", 2024-04-25T20:09:26.162 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_lc_thread_delay": "0", 2024-04-25T20:09:26.162 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-04-25T20:09:26.162 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_ldap_dnattr": "uid", 2024-04-25T20:09:26.162 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-04-25T20:09:26.162 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_ldap_searchfilter": "", 2024-04-25T20:09:26.162 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-04-25T20:09:26.162 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_ldap_uri": "ldaps://", 2024-04-25T20:09:26.162 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-04-25T20:09:26.163 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-04-25T20:09:26.163 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-04-25T20:09:26.163 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_log_http_headers": "", 2024-04-25T20:09:26.163 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_log_nonexistent_bucket": "false", 2024-04-25T20:09:26.163 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-04-25T20:09:26.163 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_log_object_name_utc": "false", 2024-04-25T20:09:26.163 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-04-25T20:09:26.163 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_max_attr_name_len": "0", 2024-04-25T20:09:26.163 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_max_attr_size": "0", 2024-04-25T20:09:26.163 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_max_attrs_num_in_req": "0", 2024-04-25T20:09:26.163 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_max_chunk_size": "4194304", 2024-04-25T20:09:26.163 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_max_concurrent_requests": "1024", 2024-04-25T20:09:26.163 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-04-25T20:09:26.164 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_max_dynamic_shards": "1999", 2024-04-25T20:09:26.164 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_max_listing_results": "1000", 2024-04-25T20:09:26.164 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_max_notify_retries": "10", 2024-04-25T20:09:26.164 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_max_objs_per_shard": "100000", 2024-04-25T20:09:26.164 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_max_put_param_size": "1048576", 2024-04-25T20:09:26.164 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_max_put_size": "5368709120", 2024-04-25T20:09:26.164 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_max_slo_entries": "1000", 2024-04-25T20:09:26.164 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_md_log_max_shards": "64", 2024-04-25T20:09:26.164 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_md_notify_interval_msec": "200", 2024-04-25T20:09:26.164 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_meta_sync_poll_interval": "20", 2024-04-25T20:09:26.164 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_meta_sync_spawn_window": "20", 2024-04-25T20:09:26.164 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-04-25T20:09:26.164 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_mp_lock_max_time": "600", 2024-04-25T20:09:26.164 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-04-25T20:09:26.165 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_multipart_min_part_size": "5242880", 2024-04-25T20:09:26.165 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-04-25T20:09:26.165 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-04-25T20:09:26.165 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_nfs_fhcache_size": "2017", 2024-04-25T20:09:26.165 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-04-25T20:09:26.165 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-04-25T20:09:26.165 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_nfs_lru_lanes": "5", 2024-04-25T20:09:26.165 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_nfs_max_gc": "300", 2024-04-25T20:09:26.165 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-04-25T20:09:26.165 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_nfs_run_gc_threads": "false", 2024-04-25T20:09:26.165 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_nfs_run_lc_threads": "false", 2024-04-25T20:09:26.165 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_nfs_run_quota_threads": "false", 2024-04-25T20:09:26.165 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_nfs_run_sync_thread": "false", 2024-04-25T20:09:26.165 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-04-25T20:09:26.166 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-04-25T20:09:26.166 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_num_async_rados_threads": "32", 2024-04-25T20:09:26.166 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_num_control_oids": "8", 2024-04-25T20:09:26.166 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_numa_node": "-1", 2024-04-25T20:09:26.166 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_obj_stripe_size": "4194304", 2024-04-25T20:09:26.166 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-04-25T20:09:26.166 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_objexp_chunk_size": "100", 2024-04-25T20:09:26.166 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_objexp_gc_interval": "600", 2024-04-25T20:09:26.166 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_objexp_hints_num_shards": "127", 2024-04-25T20:09:26.166 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-04-25T20:09:26.166 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-04-25T20:09:26.166 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_op_thread_timeout": "600", 2024-04-25T20:09:26.166 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_op_tracing": "false", 2024-04-25T20:09:26.166 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_opa_token": "", 2024-04-25T20:09:26.166 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_opa_url": "", 2024-04-25T20:09:26.167 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_opa_verify_ssl": "true", 2024-04-25T20:09:26.167 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-04-25T20:09:26.167 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-04-25T20:09:26.167 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_ops_log_rados": "false", 2024-04-25T20:09:26.167 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_ops_log_socket_path": "", 2024-04-25T20:09:26.167 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-04-25T20:09:26.167 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-04-25T20:09:26.167 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-04-25T20:09:26.167 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_period_push_interval": "2.000000", 2024-04-25T20:09:26.167 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_period_push_interval_max": "30.000000", 2024-04-25T20:09:26.167 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_period_root_pool": ".rgw.root", 2024-04-25T20:09:26.167 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-04-25T20:09:26.167 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_print_continue": "true", 2024-04-25T20:09:26.168 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_print_prohibited_content_length": "false", 2024-04-25T20:09:26.168 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-04-25T20:09:26.168 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-04-25T20:09:26.168 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-04-25T20:09:26.168 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-04-25T20:09:26.168 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_rados_tracing": "false", 2024-04-25T20:09:26.168 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_realm": "", 2024-04-25T20:09:26.168 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_realm_id": "", 2024-04-25T20:09:26.168 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-04-25T20:09:26.168 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_region": "", 2024-04-25T20:09:26.168 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_region_root_pool": ".rgw.root", 2024-04-25T20:09:26.168 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_relaxed_region_enforcement": "false", 2024-04-25T20:09:26.168 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-04-25T20:09:26.168 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-04-25T20:09:26.169 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_request_uri": "", 2024-04-25T20:09:26.169 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_reshard_batch_size": "64", 2024-04-25T20:09:26.169 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-04-25T20:09:26.169 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_reshard_max_aio": "128", 2024-04-25T20:09:26.169 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_reshard_num_logs": "16", 2024-04-25T20:09:26.169 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_reshard_thread_interval": "600", 2024-04-25T20:09:26.169 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_resolve_cname": "false", 2024-04-25T20:09:26.169 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_rest_getusage_op_compat": "false", 2024-04-25T20:09:26.169 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_run_sync_thread": "true", 2024-04-25T20:09:26.169 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-04-25T20:09:26.169 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_s3_auth_use_keystone": "false", 2024-04-25T20:09:26.169 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_s3_auth_use_ldap": "false", 2024-04-25T20:09:26.169 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_s3_auth_use_rados": "true", 2024-04-25T20:09:26.169 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_s3_auth_use_sts": "false", 2024-04-25T20:09:26.170 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-04-25T20:09:26.170 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_s3_success_create_obj_status": "0", 2024-04-25T20:09:26.170 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-04-25T20:09:26.170 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_scheduler_type": "throttler", 2024-04-25T20:09:26.170 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_script_uri": "", 2024-04-25T20:09:26.170 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_service_provider_name": "", 2024-04-25T20:09:26.170 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-04-25T20:09:26.170 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sts_client_id": "", 2024-04-25T20:09:26.170 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sts_client_secret": "", 2024-04-25T20:09:26.170 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sts_entry": "sts", 2024-04-25T20:09:26.171 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sts_key": "sts", 2024-04-25T20:09:26.171 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sts_max_session_duration": "43200", 2024-04-25T20:09:26.171 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sts_min_session_duration": "900", 2024-04-25T20:09:26.171 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sts_token_introspection_url": "", 2024-04-25T20:09:26.171 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_swift_account_in_url": "false", 2024-04-25T20:09:26.171 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_swift_auth_entry": "auth", 2024-04-25T20:09:26.171 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_swift_auth_url": "", 2024-04-25T20:09:26.171 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_swift_custom_header": "", 2024-04-25T20:09:26.171 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_swift_enforce_content_length": "false", 2024-04-25T20:09:26.171 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_swift_need_stats": "true", 2024-04-25T20:09:26.171 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_swift_tenant_name": "", 2024-04-25T20:09:26.171 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_swift_token_expiration": "86400", 2024-04-25T20:09:26.171 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_swift_url": "", 2024-04-25T20:09:26.171 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_swift_url_prefix": "swift", 2024-04-25T20:09:26.171 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_swift_versioning_enabled": "false", 2024-04-25T20:09:26.172 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-04-25T20:09:26.172 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-04-25T20:09:26.172 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sync_lease_period": "120", 2024-04-25T20:09:26.172 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-04-25T20:09:26.172 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sync_log_trim_interval": "1200", 2024-04-25T20:09:26.172 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-04-25T20:09:26.172 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-04-25T20:09:26.172 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-04-25T20:09:26.172 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sync_obj_etag_verify": "false", 2024-04-25T20:09:26.172 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sync_trace_history_size": "4096", 2024-04-25T20:09:26.172 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-04-25T20:09:26.172 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-04-25T20:09:26.172 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_thread_pool_size": "512", 2024-04-25T20:09:26.172 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_torrent_comment": "", 2024-04-25T20:09:26.172 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_torrent_createby": "", 2024-04-25T20:09:26.173 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_torrent_encoding": "", 2024-04-25T20:09:26.173 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_torrent_flag": "false", 2024-04-25T20:09:26.173 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_torrent_origin": "", 2024-04-25T20:09:26.173 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_torrent_sha_unit": "524288", 2024-04-25T20:09:26.173 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_torrent_tracker": "", 2024-04-25T20:09:26.173 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_trust_forwarded_https": "false", 2024-04-25T20:09:26.173 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-04-25T20:09:26.173 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_usage_log_tick_interval": "30", 2024-04-25T20:09:26.173 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_usage_max_shards": "32", 2024-04-25T20:09:26.173 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_usage_max_user_shards": "1", 2024-04-25T20:09:26.173 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_use_opa_authz": "false", 2024-04-25T20:09:26.173 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-04-25T20:09:26.173 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_user_default_quota_max_size": "-1", 2024-04-25T20:09:26.173 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_user_max_buckets": "1000", 2024-04-25T20:09:26.173 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_user_policies_max_num": "100", 2024-04-25T20:09:26.173 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-04-25T20:09:26.174 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-04-25T20:09:26.174 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_user_quota_sync_interval": "86400", 2024-04-25T20:09:26.174 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-04-25T20:09:26.174 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_user_unique_email": "true", 2024-04-25T20:09:26.174 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_verify_ssl": "true", 2024-04-25T20:09:26.174 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_website_routing_rules_max_num": "50", 2024-04-25T20:09:26.174 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_zone": "", 2024-04-25T20:09:26.174 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_zone_id": "", 2024-04-25T20:09:26.174 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-04-25T20:09:26.174 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_zonegroup": "", 2024-04-25T20:09:26.174 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_zonegroup_id": "", 2024-04-25T20:09:26.174 INFO:teuthology.orchestra.run.smithi039.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-04-25T20:09:26.174 INFO:teuthology.orchestra.run.smithi039.stdout: "rgwlc_auto_session_clear": "true", 2024-04-25T20:09:26.174 INFO:teuthology.orchestra.run.smithi039.stdout: "rgwlc_skip_bucket_step": "false", 2024-04-25T20:09:26.175 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_block_size": "4096", 2024-04-25T20:09:26.175 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-04-25T20:09:26.175 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-04-25T20:09:26.175 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-04-25T20:09:26.175 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-04-25T20:09:26.175 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_cache_shard_bits": "4", 2024-04-25T20:09:26.175 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_cache_size": "536870912", 2024-04-25T20:09:26.175 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_cache_type": "binned_lru", 2024-04-25T20:09:26.175 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-04-25T20:09:26.175 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-04-25T20:09:26.175 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-04-25T20:09:26.175 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_collect_compaction_stats": "false", 2024-04-25T20:09:26.175 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_collect_extended_stats": "false", 2024-04-25T20:09:26.175 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_collect_memory_stats": "false", 2024-04-25T20:09:26.175 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-04-25T20:09:26.176 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_index_type": "binary_search", 2024-04-25T20:09:26.176 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_log_to_ceph_log": "true", 2024-04-25T20:09:26.176 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_metadata_block_size": "4096", 2024-04-25T20:09:26.176 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_partition_filters": "false", 2024-04-25T20:09:26.176 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_perf": "false", 2024-04-25T20:09:26.176 INFO:teuthology.orchestra.run.smithi039.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-04-25T20:09:26.176 INFO:teuthology.orchestra.run.smithi039.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-04-25T20:09:26.176 INFO:teuthology.orchestra.run.smithi039.stdout: "rotating_keys_renewal_timeout": "10", 2024-04-25T20:09:26.176 INFO:teuthology.orchestra.run.smithi039.stdout: "run_dir": "/var/run/ceph", 2024-04-25T20:09:26.176 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_block_create": "true", 2024-04-25T20:09:26.176 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_cache_lru_size": "67108864", 2024-04-25T20:09:26.176 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_cbjournal_size": "5368709120", 2024-04-25T20:09:26.176 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_default_max_object_size": "16777216", 2024-04-25T20:09:26.176 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-04-25T20:09:26.176 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_device_size": "53687091200", 2024-04-25T20:09:26.177 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_journal_batch_capacity": "16", 2024-04-25T20:09:26.177 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-04-25T20:09:26.177 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-04-25T20:09:26.177 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_journal_iodepth_limit": "5", 2024-04-25T20:09:26.177 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_main_device_type": "SSD", 2024-04-25T20:09:26.177 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_max_concurrent_transactions": "8", 2024-04-25T20:09:26.177 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-04-25T20:09:26.177 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-04-25T20:09:26.177 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-04-25T20:09:26.177 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-04-25T20:09:26.177 INFO:teuthology.orchestra.run.smithi039.stdout: "seastore_segment_size": "67108864", 2024-04-25T20:09:26.177 INFO:teuthology.orchestra.run.smithi039.stdout: "set_keepcaps": "false", 2024-04-25T20:09:26.177 INFO:teuthology.orchestra.run.smithi039.stdout: "setgroup": "", 2024-04-25T20:09:26.177 INFO:teuthology.orchestra.run.smithi039.stdout: "setuser": "", 2024-04-25T20:09:26.177 INFO:teuthology.orchestra.run.smithi039.stdout: "setuser_match_path": "", 2024-04-25T20:09:26.178 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_misplaced_ratio": "0.050000", 2024-04-25T20:09:26.178 INFO:teuthology.orchestra.run.smithi039.stdout: "thp": "false", 2024-04-25T20:09:26.178 INFO:teuthology.orchestra.run.smithi039.stdout: "threadpool_default_timeout": "60", 2024-04-25T20:09:26.178 INFO:teuthology.orchestra.run.smithi039.stdout: "threadpool_empty_queue_max_wait": "2", 2024-04-25T20:09:26.178 INFO:teuthology.orchestra.run.smithi039.stdout: "throttler_perf_counter": "true" 2024-04-25T20:09:26.178 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:26.180 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-04-25T20:09:26.295 INFO:teuthology.orchestra.run.smithi039.stdout:mon.a: {} 2024-04-25T20:09:26.295 INFO:teuthology.orchestra.run.smithi039.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-25T20:09:26.304 INFO:teuthology.orchestra.run.smithi039.stdout:mon.b: {} 2024-04-25T20:09:26.304 INFO:teuthology.orchestra.run.smithi039.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-25T20:09:26.313 INFO:teuthology.orchestra.run.smithi039.stdout:mon.c: {} 2024-04-25T20:09:26.313 INFO:teuthology.orchestra.run.smithi039.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-25T20:09:26.324 DEBUG:teuthology.orchestra.run.smithi135:> type ceph-objectstore-tool 2024-04-25T20:09:26.342 INFO:teuthology.orchestra.run.smithi135.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-25T20:09:26.342 DEBUG:teuthology.orchestra.run.smithi039:> type ceph-objectstore-tool 2024-04-25T20:09:26.356 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-25T20:09:26.357 INFO:teuthology.run_tasks:Running task workunit... 2024-04-25T20:09:26.369 INFO:tasks.workunit:Pulling workunits from ref 19b26365dac54ee709df8c5dc2a7b7e89a50aba5 2024-04-25T20:09:26.370 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T20:09:26.370 DEBUG:teuthology.orchestra.run.smithi039:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T20:09:26.370 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-04-25T20:09:26.370 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-25T20:09:26.371 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T20:09:26.371 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-04-25T20:09:26.371 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 3.0 2024-04-25T20:09:26.371 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-04-25T20:09:26.372 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-04-25T20:09:26.372 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T20:09:26.372 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-04-25T20:09:26.372 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T20:09:26.373 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-04-25T20:09:26.373 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-25T20:09:26.384 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:26.384+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:26.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:09:26.415 INFO:teuthology.orchestra.run.smithi039.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T20:09:26.415 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-04-25T20:09:26.473 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:26.474+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:26.477 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-04-25T20:09:26.477 DEBUG:teuthology.orchestra.run.smithi039:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-04-25T20:09:26.520 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:26.520 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:26.520 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:09:26.520 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:26.531 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T20:09:26.535 INFO:tasks.workunit:timeout=3h 2024-04-25T20:09:26.535 INFO:tasks.workunit:cleanup=True 2024-04-25T20:09:26.535 DEBUG:teuthology.orchestra.run.smithi039:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 19b26365dac54ee709df8c5dc2a7b7e89a50aba5 2024-04-25T20:09:26.538 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:09:26.539 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:09:26.558 INFO:tasks.workunit.client.0.smithi039.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T20:09:26.563 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:09:26.563 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:09:26.575 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:26.575+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:26.586 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:09:26.586 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:09:26.620 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:09:26.620 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:09:26.642 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:09:26.642 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:09:26.651 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:26.652 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:26.659 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:09:26.659 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:09:26.661 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T20:09:26.675 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:26.675+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:26.676 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:09:26.677 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:09:26.695 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:09:26.695 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:09:26.720 INFO:teuthology.orchestra.run.smithi039.stdout:moved 0 / 18 (0%) 2024-04-25T20:09:26.720 INFO:teuthology.orchestra.run.smithi039.stdout:avg 2.25 2024-04-25T20:09:26.720 INFO:teuthology.orchestra.run.smithi039.stdout:stddev 0.968246 -> 0.968246 (expected baseline 1.40312) 2024-04-25T20:09:26.720 INFO:teuthology.orchestra.run.smithi039.stdout:min osd.2 with 1 -> 1 pgs (0.444444 -> 0.444444 * mean) 2024-04-25T20:09:26.720 INFO:teuthology.orchestra.run.smithi039.stdout:max osd.0 with 4 -> 4 pgs (1.77778 -> 1.77778 * mean) 2024-04-25T20:09:26.720 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:09:26.720 INFO:teuthology.orchestra.run.smithi039.stdout:oload 110 2024-04-25T20:09:26.720 INFO:teuthology.orchestra.run.smithi039.stdout:max_change 3 2024-04-25T20:09:26.721 INFO:teuthology.orchestra.run.smithi039.stdout:max_change_osds 4 2024-04-25T20:09:26.721 INFO:teuthology.orchestra.run.smithi039.stdout:average_utilization 0.0003 2024-04-25T20:09:26.721 INFO:teuthology.orchestra.run.smithi039.stdout:overload_utilization 0.0003 2024-04-25T20:09:26.721 INFO:teuthology.orchestra.run.smithi039.stderr:no change 2024-04-25T20:09:26.733 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-04-25T20:09:26.733 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 1.0 2024-04-25T20:09:26.776 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:26.776+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:26.786 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:26.786 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:26.796 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T20:09:26.881 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:26.880+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:26.945 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:26.945 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:26.954 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T20:09:26.995 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:26.994+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:27.025 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is set 2024-04-25T20:09:27.072 INFO:teuthology.orchestra.run.smithi039.stdout:moved 0 / 18 (0%) 2024-04-25T20:09:27.072 INFO:teuthology.orchestra.run.smithi039.stdout:avg 2.25 2024-04-25T20:09:27.072 INFO:teuthology.orchestra.run.smithi039.stdout:stddev 0.968246 -> 0.968246 (expected baseline 1.40312) 2024-04-25T20:09:27.072 INFO:teuthology.orchestra.run.smithi039.stdout:min osd.2 with 1 -> 1 pgs (0.444444 -> 0.444444 * mean) 2024-04-25T20:09:27.073 INFO:teuthology.orchestra.run.smithi039.stdout:max osd.0 with 4 -> 4 pgs (1.77778 -> 1.77778 * mean) 2024-04-25T20:09:27.073 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:09:27.073 INFO:teuthology.orchestra.run.smithi039.stdout:oload 1000 2024-04-25T20:09:27.073 INFO:teuthology.orchestra.run.smithi039.stdout:max_change 1 2024-04-25T20:09:27.073 INFO:teuthology.orchestra.run.smithi039.stdout:max_change_osds 4 2024-04-25T20:09:27.073 INFO:teuthology.orchestra.run.smithi039.stdout:average_utilization 0.0003 2024-04-25T20:09:27.073 INFO:teuthology.orchestra.run.smithi039.stdout:overload_utilization 0.0031 2024-04-25T20:09:27.073 INFO:teuthology.orchestra.run.smithi039.stderr:no change 2024-04-25T20:09:27.078 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:27.077+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:27.086 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-04-25T20:09:27.086 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 1.0 2024-04-25T20:09:27.106 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:27.106 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:27.116 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T20:09:27.180 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:27.180+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:27.265 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:27.265 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:27.274 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T20:09:27.318 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:27.317+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:27.381 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:27.382 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:27.382 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:27.382+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:27.390 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T20:09:27.411 INFO:teuthology.orchestra.run.smithi039.stdout:moved 0 / 18 (0%) 2024-04-25T20:09:27.411 INFO:teuthology.orchestra.run.smithi039.stdout:avg 2.25 2024-04-25T20:09:27.412 INFO:teuthology.orchestra.run.smithi039.stdout:stddev 0.968246 -> 0.968246 (expected baseline 1.40312) 2024-04-25T20:09:27.412 INFO:teuthology.orchestra.run.smithi039.stdout:min osd.2 with 1 -> 1 pgs (0.444444 -> 0.444444 * mean) 2024-04-25T20:09:27.412 INFO:teuthology.orchestra.run.smithi039.stdout:max osd.0 with 4 -> 4 pgs (1.77778 -> 1.77778 * mean) 2024-04-25T20:09:27.412 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:09:27.412 INFO:teuthology.orchestra.run.smithi039.stdout:oload 110 2024-04-25T20:09:27.412 INFO:teuthology.orchestra.run.smithi039.stdout:max_change 1 2024-04-25T20:09:27.412 INFO:teuthology.orchestra.run.smithi039.stdout:max_change_osds 4 2024-04-25T20:09:27.412 INFO:teuthology.orchestra.run.smithi039.stdout:average_utilization 0.0003 2024-04-25T20:09:27.412 INFO:teuthology.orchestra.run.smithi039.stdout:overload_utilization 0.0003 2024-04-25T20:09:27.412 INFO:teuthology.orchestra.run.smithi039.stderr:no change 2024-04-25T20:09:27.424 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-04-25T20:09:27.424 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2024-04-25T20:09:27.483 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:27.483+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:27.519 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:27.519 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:27.528 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T20:09:27.583 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:27.582+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:27.638 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:27.638 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:27.647 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T20:09:27.689 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:27.687+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:27.761 INFO:teuthology.orchestra.run.smithi039.stdout:moved 0 / 18 (0%) 2024-04-25T20:09:27.761 INFO:teuthology.orchestra.run.smithi039.stdout:avg 2.25 2024-04-25T20:09:27.761 INFO:teuthology.orchestra.run.smithi039.stdout:stddev 0.968246 -> 0.968246 (expected baseline 1.40312) 2024-04-25T20:09:27.761 INFO:teuthology.orchestra.run.smithi039.stdout:min osd.2 with 1 -> 1 pgs (0.444444 -> 0.444444 * mean) 2024-04-25T20:09:27.761 INFO:teuthology.orchestra.run.smithi039.stdout:max osd.0 with 4 -> 4 pgs (1.77778 -> 1.77778 * mean) 2024-04-25T20:09:27.762 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:09:27.762 INFO:teuthology.orchestra.run.smithi039.stdout:oload 1000 2024-04-25T20:09:27.762 INFO:teuthology.orchestra.run.smithi039.stdout:max_change 3 2024-04-25T20:09:27.762 INFO:teuthology.orchestra.run.smithi039.stdout:max_change_osds 4 2024-04-25T20:09:27.762 INFO:teuthology.orchestra.run.smithi039.stdout:average_utilization 0.0003 2024-04-25T20:09:27.762 INFO:teuthology.orchestra.run.smithi039.stdout:overload_utilization 0.0031 2024-04-25T20:09:27.763 INFO:teuthology.orchestra.run.smithi039.stderr:no change 2024-04-25T20:09:27.774 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-04-25T20:09:27.774 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 1.0 2024-04-25T20:09:27.785 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:27.785+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:27.795 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:27.796 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:27.805 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T20:09:27.886 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:27.886+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:27.954 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:27.954 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:27.963 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T20:09:27.987 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:27.987+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:28.071 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:28.071 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:28.073 INFO:teuthology.orchestra.run.smithi039.stdout:moved 0 / 18 (0%) 2024-04-25T20:09:28.073 INFO:teuthology.orchestra.run.smithi039.stdout:avg 2.25 2024-04-25T20:09:28.073 INFO:teuthology.orchestra.run.smithi039.stdout:stddev 0.968246 -> 0.968246 (expected baseline 1.40312) 2024-04-25T20:09:28.073 INFO:teuthology.orchestra.run.smithi039.stdout:min osd.2 with 1 -> 1 pgs (0.444444 -> 0.444444 * mean) 2024-04-25T20:09:28.073 INFO:teuthology.orchestra.run.smithi039.stdout:max osd.0 with 4 -> 4 pgs (1.77778 -> 1.77778 * mean) 2024-04-25T20:09:28.073 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:09:28.073 INFO:teuthology.orchestra.run.smithi039.stdout:oload 1000 2024-04-25T20:09:28.074 INFO:teuthology.orchestra.run.smithi039.stdout:max_change 1 2024-04-25T20:09:28.074 INFO:teuthology.orchestra.run.smithi039.stdout:max_change_osds 4 2024-04-25T20:09:28.074 INFO:teuthology.orchestra.run.smithi039.stdout:average_utilization 0.0003 2024-04-25T20:09:28.074 INFO:teuthology.orchestra.run.smithi039.stdout:overload_utilization 0.0031 2024-04-25T20:09:28.074 INFO:teuthology.orchestra.run.smithi039.stderr:no change 2024-04-25T20:09:28.081 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T20:09:28.126 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:28.126+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:28.188 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:28.188+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:28.195 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:28.195 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:28.204 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T20:09:28.288 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:28.287+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:28.346 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:28.346 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:28.355 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T20:09:28.390 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:28.388+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:28.456 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:28.456 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:28.466 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T20:09:28.491 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:28.490+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:28.566 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:28.567 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:28.575 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T20:09:28.591 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:28.590+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:28.675 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:28.675 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:28.684 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T20:09:28.707 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T20:09:28.728 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:28.728+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:28.785 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:28.785 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:28.793 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T20:09:28.794 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:28.793+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:28.831 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:09:28.831 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:09:28.848 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:09:28.849 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:09:28.865 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:09:28.865 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:09:28.881 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:09:28.881 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:09:28.893 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:28.894+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:28.899 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:09:28.899 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:09:28.917 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:09:28.917 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:09:28.934 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:09:28.935 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:09:28.938 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:28.938 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:28.938 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:09:28.938 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:28.947 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T20:09:28.951 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:09:28.951 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:09:28.994 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:28.993+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:29.041 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-25T20:09:29.089 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:29.089 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:29.089 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:09:29.089 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:09:29.089 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:29.095 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:29.094+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:29.097 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T20:09:29.197 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:29.196+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:29.199 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:29.199 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:09:29.199 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:09:29.199 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:09:29.199 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:29.208 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T20:09:29.298 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:29.298+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:29.310 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:29.310 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:29.310 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:09:29.310 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:29.318 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T20:09:29.399 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:29.397+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:29.460 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:29.460 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:29.461 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:09:29.461 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:09:29.461 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:29.469 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T20:09:29.500 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:29.499+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:29.573 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:29.573 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:09:29.573 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:09:29.573 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [ 2024-04-25T20:09:29.573 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.573 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:09:29.573 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:09:29.573 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 12.788844743, 2024-04-25T20:09:29.573 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.34154860100000001, 2024-04-25T20:09:29.574 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:29.574 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:29.574 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:29.574 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:29.574 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:29.574 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 1 2024-04-25T20:09:29.574 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.574 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:29.574 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.574 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:29.574 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:09:29.574 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.574 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.575 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.575 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:29.575 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:09:29.575 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.575 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.575 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.575 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:29.575 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782261+0000", 2024-04-25T20:09:29.575 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999986 2024-04-25T20:09:29.575 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.575 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.575 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:29.575 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782268+0000", 2024-04-25T20:09:29.575 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.8140000000000001e-06 2024-04-25T20:09:29.576 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.576 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.576 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:29.576 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782279+0000", 2024-04-25T20:09:29.576 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0665999999999999e-05 2024-04-25T20:09:29.576 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.576 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.576 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:29.576 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782290+0000", 2024-04-25T20:09:29.576 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.1343e-05 2024-04-25T20:09:29.576 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.576 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.576 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:29.576 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782340+0000", 2024-04-25T20:09:29.576 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.9311999999999998e-05 2024-04-25T20:09:29.577 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.577 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.577 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for peered", 2024-04-25T20:09:29.577 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782348+0000", 2024-04-25T20:09:29.577 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.0690000000000002e-06 2024-04-25T20:09:29.577 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.577 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.577 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:29.577 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120520+0000", 2024-04-25T20:09:29.577 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.33817188100000001 2024-04-25T20:09:29.577 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.577 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.577 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:29.577 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120656+0000", 2024-04-25T20:09:29.578 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000135897 2024-04-25T20:09:29.578 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.578 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.578 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:29.578 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120802+0000", 2024-04-25T20:09:29.578 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00014691199999999999 2024-04-25T20:09:29.578 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.578 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.579 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:29.579 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.122368+0000", 2024-04-25T20:09:29.579 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0015651040000000001 2024-04-25T20:09:29.579 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.579 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.579 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:29.579 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123729+0000", 2024-04-25T20:09:29.579 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0013610499999999999 2024-04-25T20:09:29.579 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.579 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.579 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:29.579 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123769+0000", 2024-04-25T20:09:29.579 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0719000000000001e-05 2024-04-25T20:09:29.580 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.580 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.580 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:29.580 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123811+0000", 2024-04-25T20:09:29.580 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1267999999999999e-05 2024-04-25T20:09:29.580 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.580 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:29.580 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.580 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.580 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.580 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:67 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:09:29.581 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:09:29.581 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 12.359425856, 2024-04-25T20:09:29.581 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.006947651, 2024-04-25T20:09:29.581 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:29.581 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:29.581 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:29.581 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:29.581 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:29.581 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 67 2024-04-25T20:09:29.581 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.581 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:29.581 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.581 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:29.581 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:09:29.582 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.582 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.582 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.582 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:29.582 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:09:29.582 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.582 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.582 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.582 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:29.582 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211680+0000", 2024-04-25T20:09:29.582 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:29.582 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.582 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.582 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:29.583 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211684+0000", 2024-04-25T20:09:29.583 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4249999999999998e-06 2024-04-25T20:09:29.583 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.583 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.583 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:29.583 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211690+0000", 2024-04-25T20:09:29.583 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.7360000000000004e-06 2024-04-25T20:09:29.583 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.583 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.583 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:29.583 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211694+0000", 2024-04-25T20:09:29.583 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4860000000000002e-06 2024-04-25T20:09:29.583 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.583 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.584 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:29.584 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211727+0000", 2024-04-25T20:09:29.584 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3445999999999998e-05 2024-04-25T20:09:29.584 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.584 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.584 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:29.584 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211778+0000", 2024-04-25T20:09:29.584 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1050000000000001e-05 2024-04-25T20:09:29.584 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.584 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.584 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:29.584 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211851+0000", 2024-04-25T20:09:29.585 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2914e-05 2024-04-25T20:09:29.585 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.585 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.585 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:29.585 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213824+0000", 2024-04-25T20:09:29.585 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0019726520000000001 2024-04-25T20:09:29.586 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.586 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.586 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:29.586 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218579+0000", 2024-04-25T20:09:29.586 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0047553159999999999 2024-04-25T20:09:29.586 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.586 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.586 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:29.586 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218607+0000", 2024-04-25T20:09:29.587 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8144999999999999e-05 2024-04-25T20:09:29.587 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.587 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.587 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:29.587 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218629+0000", 2024-04-25T20:09:29.587 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.1044e-05 2024-04-25T20:09:29.587 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.587 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:29.587 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.587 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.588 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.588 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:68 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:09:29.588 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:09:29.588 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 12.359407515000001, 2024-04-25T20:09:29.588 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012255596000000001, 2024-04-25T20:09:29.588 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:29.588 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:29.588 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:29.588 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:29.589 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:29.589 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 68 2024-04-25T20:09:29.589 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.589 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:29.589 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.589 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:29.589 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:09:29.589 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.589 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.589 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.589 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:29.589 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:09:29.589 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.589 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.590 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.590 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:29.590 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:09:29.590 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:29.590 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.590 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.590 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:29.590 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211701+0000", 2024-04-25T20:09:29.590 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2029999999999999e-06 2024-04-25T20:09:29.590 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.590 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.590 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:29.590 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211705+0000", 2024-04-25T20:09:29.590 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.2320000000000003e-06 2024-04-25T20:09:29.591 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.591 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.591 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:29.591 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211708+0000", 2024-04-25T20:09:29.591 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.6290000000000002e-06 2024-04-25T20:09:29.591 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.591 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.591 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:29.591 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212136+0000", 2024-04-25T20:09:29.591 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00042764099999999998 2024-04-25T20:09:29.591 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.591 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.591 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:29.591 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212212+0000", 2024-04-25T20:09:29.592 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.6315000000000004e-05 2024-04-25T20:09:29.592 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.592 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.592 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:29.592 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212306+0000", 2024-04-25T20:09:29.592 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.3556999999999999e-05 2024-04-25T20:09:29.592 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.592 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.592 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:29.592 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213829+0000", 2024-04-25T20:09:29.592 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001523256 2024-04-25T20:09:29.592 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.592 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.592 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:29.592 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223904+0000", 2024-04-25T20:09:29.593 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010075387 2024-04-25T20:09:29.593 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.593 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.593 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:29.593 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223934+0000", 2024-04-25T20:09:29.593 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9988000000000002e-05 2024-04-25T20:09:29.593 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.593 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.593 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:29.593 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223955+0000", 2024-04-25T20:09:29.593 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0630999999999999e-05 2024-04-25T20:09:29.593 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.593 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:29.593 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.594 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.594 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.594 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:69 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:09:29.594 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:09:29.594 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 12.35938578, 2024-04-25T20:09:29.594 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012316863000000001, 2024-04-25T20:09:29.594 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:29.594 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:29.594 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:29.594 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:29.594 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:29.594 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 69 2024-04-25T20:09:29.594 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.594 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:29.595 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.595 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:29.595 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:09:29.595 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.595 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.595 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.595 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:29.595 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:09:29.595 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.595 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.595 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.595 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:29.595 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:09:29.595 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:29.595 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.596 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.596 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:29.596 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211762+0000", 2024-04-25T20:09:29.596 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1703999999999999e-05 2024-04-25T20:09:29.596 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.596 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.596 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:29.596 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211768+0000", 2024-04-25T20:09:29.596 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.3430000000000003e-06 2024-04-25T20:09:29.596 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.596 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.596 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:29.596 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211771+0000", 2024-04-25T20:09:29.596 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.18e-06 2024-04-25T20:09:29.597 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.597 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.597 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:29.597 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212652+0000", 2024-04-25T20:09:29.597 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00088095800000000004 2024-04-25T20:09:29.597 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.597 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.597 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:29.597 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212703+0000", 2024-04-25T20:09:29.597 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.0655999999999997e-05 2024-04-25T20:09:29.597 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.597 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.597 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:29.597 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212781+0000", 2024-04-25T20:09:29.597 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.8852999999999999e-05 2024-04-25T20:09:29.598 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.598 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.598 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:29.598 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.214131+0000", 2024-04-25T20:09:29.598 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0013496039999999999 2024-04-25T20:09:29.598 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.598 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.598 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:29.598 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223986+0000", 2024-04-25T20:09:29.598 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.009854804 2024-04-25T20:09:29.598 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.598 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.598 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:29.599 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.224017+0000", 2024-04-25T20:09:29.599 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.1068999999999997e-05 2024-04-25T20:09:29.599 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.599 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.599 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:29.599 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.224038+0000", 2024-04-25T20:09:29.599 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0965e-05 2024-04-25T20:09:29.599 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.599 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:29.600 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.600 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.600 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.600 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:70 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:09:29.600 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:09:29.600 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 12.359330698999999, 2024-04-25T20:09:29.600 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.015776348999999999, 2024-04-25T20:09:29.600 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:29.600 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:29.600 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:29.600 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:29.600 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:29.600 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 70 2024-04-25T20:09:29.600 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.600 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:29.601 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.601 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:29.601 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:09:29.601 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.601 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.601 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.601 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:29.601 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:09:29.601 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.601 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.601 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.601 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:29.601 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:09:29.601 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:29.602 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.602 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.602 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:29.602 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211778+0000", 2024-04-25T20:09:29.602 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5330000000000001e-06 2024-04-25T20:09:29.602 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.602 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.602 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:29.602 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211782+0000", 2024-04-25T20:09:29.602 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4079999999999998e-06 2024-04-25T20:09:29.602 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.602 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.602 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:29.603 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211786+0000", 2024-04-25T20:09:29.603 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.6190000000000002e-06 2024-04-25T20:09:29.603 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.603 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.603 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:29.603 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213374+0000", 2024-04-25T20:09:29.603 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0015881859999999999 2024-04-25T20:09:29.603 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.603 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.603 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:29.603 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213478+0000", 2024-04-25T20:09:29.603 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000103464 2024-04-25T20:09:29.603 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.603 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.603 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:29.604 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213601+0000", 2024-04-25T20:09:29.604 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00012324300000000001 2024-04-25T20:09:29.604 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.604 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.604 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:29.604 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.214788+0000", 2024-04-25T20:09:29.604 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011868569999999999 2024-04-25T20:09:29.604 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.604 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.604 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:29.604 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227504+0000", 2024-04-25T20:09:29.604 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012715749 2024-04-25T20:09:29.604 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.604 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.605 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:29.605 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227532+0000", 2024-04-25T20:09:29.605 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.7937e-05 2024-04-25T20:09:29.605 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.605 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.605 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:29.605 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227552+0000", 2024-04-25T20:09:29.605 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0772999999999999e-05 2024-04-25T20:09:29.605 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.605 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:29.605 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.605 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.605 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.605 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:131 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:29.605 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:09:29.606 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 11.450320853999999, 2024-04-25T20:09:29.606 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0086158810000000006, 2024-04-25T20:09:29.606 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:29.606 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:29.606 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:29.606 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:29.606 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:29.606 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 131 2024-04-25T20:09:29.606 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.606 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:29.606 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.606 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:29.606 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:09:29.607 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.607 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.607 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.607 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:29.607 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:09:29.607 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.607 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.607 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.607 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:29.607 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120785+0000", 2024-04-25T20:09:29.607 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:29.607 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.607 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.607 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:29.607 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120880+0000", 2024-04-25T20:09:29.608 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.4925000000000003e-05 2024-04-25T20:09:29.608 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.608 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.608 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:29.608 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120891+0000", 2024-04-25T20:09:29.608 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0915e-05 2024-04-25T20:09:29.608 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.608 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.608 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:29.608 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120895+0000", 2024-04-25T20:09:29.608 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.8070000000000001e-06 2024-04-25T20:09:29.608 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.608 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.608 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:29.608 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121640+0000", 2024-04-25T20:09:29.609 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00074468500000000003 2024-04-25T20:09:29.609 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.609 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.609 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:29.609 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121684+0000", 2024-04-25T20:09:29.609 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4452000000000001e-05 2024-04-25T20:09:29.609 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.609 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.609 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:29.609 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121753+0000", 2024-04-25T20:09:29.609 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8508000000000002e-05 2024-04-25T20:09:29.609 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.609 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.609 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:29.609 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123618+0000", 2024-04-25T20:09:29.610 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0018655099999999999 2024-04-25T20:09:29.610 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.610 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.610 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:29.610 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129248+0000", 2024-04-25T20:09:29.610 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0056295540000000002 2024-04-25T20:09:29.610 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.610 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.610 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:29.610 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129368+0000", 2024-04-25T20:09:29.611 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000120244 2024-04-25T20:09:29.611 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.611 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.611 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:29.611 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129402+0000", 2024-04-25T20:09:29.611 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3812000000000001e-05 2024-04-25T20:09:29.611 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.611 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:29.611 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.611 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.611 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.611 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:132 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:29.611 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:09:29.612 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 11.450206644, 2024-04-25T20:09:29.612 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0085898169999999996, 2024-04-25T20:09:29.612 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:29.612 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:29.612 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:29.612 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:29.612 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:29.612 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 132 2024-04-25T20:09:29.612 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.612 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:29.612 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.612 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:29.612 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:09:29.612 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.612 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.613 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.613 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:29.613 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:09:29.613 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.613 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.613 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.613 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:29.613 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:09:29.613 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:29.613 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.613 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.613 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:29.613 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120902+0000", 2024-04-25T20:09:29.613 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.6129999999999998e-06 2024-04-25T20:09:29.613 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.614 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.614 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:29.614 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120906+0000", 2024-04-25T20:09:29.614 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6569999999999999e-06 2024-04-25T20:09:29.614 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.614 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.614 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:29.614 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120909+0000", 2024-04-25T20:09:29.614 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5550000000000001e-06 2024-04-25T20:09:29.614 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.614 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.614 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:29.615 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122098+0000", 2024-04-25T20:09:29.615 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011898049999999999 2024-04-25T20:09:29.615 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.615 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.615 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:29.615 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122172+0000", 2024-04-25T20:09:29.615 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3917000000000004e-05 2024-04-25T20:09:29.615 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.615 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.615 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:29.615 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122288+0000", 2024-04-25T20:09:29.615 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00011566 2024-04-25T20:09:29.616 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.616 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.616 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:29.616 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124519+0000", 2024-04-25T20:09:29.616 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0022308359999999999 2024-04-25T20:09:29.616 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.616 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.616 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:29.616 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129414+0000", 2024-04-25T20:09:29.616 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048953720000000003 2024-04-25T20:09:29.616 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.617 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.617 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:29.617 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129458+0000", 2024-04-25T20:09:29.617 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.3426000000000001e-05 2024-04-25T20:09:29.617 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.617 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.617 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:29.617 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129490+0000", 2024-04-25T20:09:29.617 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2329999999999997e-05 2024-04-25T20:09:29.617 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.617 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:29.617 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.618 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.618 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.618 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:133 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:29.618 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:09:29.618 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 11.450152774999999, 2024-04-25T20:09:29.618 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0086066270000000004, 2024-04-25T20:09:29.618 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:29.618 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:29.618 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:29.618 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:29.618 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:29.618 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 133 2024-04-25T20:09:29.619 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.619 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:29.619 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.619 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:29.619 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:09:29.619 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.619 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.619 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.619 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:29.619 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:09:29.619 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.619 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.619 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.619 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:29.619 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:09:29.620 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:29.620 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.620 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.620 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:29.620 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120956+0000", 2024-04-25T20:09:29.620 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5289999999999998e-06 2024-04-25T20:09:29.620 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.620 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.620 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:29.620 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120962+0000", 2024-04-25T20:09:29.620 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.9490000000000001e-06 2024-04-25T20:09:29.620 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.620 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.620 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:29.621 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120965+0000", 2024-04-25T20:09:29.621 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.1860000000000001e-06 2024-04-25T20:09:29.621 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.621 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.621 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:29.621 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122666+0000", 2024-04-25T20:09:29.621 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0017004500000000001 2024-04-25T20:09:29.621 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.621 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.622 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:29.622 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122741+0000", 2024-04-25T20:09:29.622 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.5297999999999996e-05 2024-04-25T20:09:29.622 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.622 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.622 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:29.622 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122858+0000", 2024-04-25T20:09:29.622 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00011723899999999999 2024-04-25T20:09:29.622 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.622 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.622 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:29.622 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124625+0000", 2024-04-25T20:09:29.622 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0017670100000000001 2024-04-25T20:09:29.623 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.623 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.623 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:29.623 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129497+0000", 2024-04-25T20:09:29.623 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048713580000000001 2024-04-25T20:09:29.623 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.623 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.623 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:29.623 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129532+0000", 2024-04-25T20:09:29.623 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5704999999999998e-05 2024-04-25T20:09:29.623 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.623 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.623 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:29.623 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129561+0000", 2024-04-25T20:09:29.624 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8283e-05 2024-04-25T20:09:29.624 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.624 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:29.624 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.624 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.624 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.624 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:134 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:29.624 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:09:29.624 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 11.450050577000001, 2024-04-25T20:09:29.624 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0085771179999999999, 2024-04-25T20:09:29.624 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:29.624 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:29.624 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:29.624 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:29.624 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:29.625 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 134 2024-04-25T20:09:29.625 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.625 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:29.625 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.625 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:29.625 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:09:29.625 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.625 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.625 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.625 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:29.625 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:09:29.625 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.625 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.625 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.626 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:29.626 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:09:29.626 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:29.626 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.626 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.626 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:29.626 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121157+0000", 2024-04-25T20:09:29.626 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00010104599999999999 2024-04-25T20:09:29.626 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.626 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.626 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:29.626 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121189+0000", 2024-04-25T20:09:29.626 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2589000000000003e-05 2024-04-25T20:09:29.626 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.627 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.627 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:29.627 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121195+0000", 2024-04-25T20:09:29.627 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.083e-06 2024-04-25T20:09:29.627 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.627 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.627 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:29.627 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123136+0000", 2024-04-25T20:09:29.627 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0019409780000000001 2024-04-25T20:09:29.627 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.627 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.627 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:29.627 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123181+0000", 2024-04-25T20:09:29.627 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4267999999999997e-05 2024-04-25T20:09:29.628 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.628 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.628 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:29.628 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123247+0000", 2024-04-25T20:09:29.628 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.6086000000000004e-05 2024-04-25T20:09:29.628 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.628 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.628 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:29.628 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125639+0000", 2024-04-25T20:09:29.628 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.002392 2024-04-25T20:09:29.628 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.628 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.628 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:29.628 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129567+0000", 2024-04-25T20:09:29.629 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0039282170000000003 2024-04-25T20:09:29.629 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.629 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.629 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:29.629 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129605+0000", 2024-04-25T20:09:29.629 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.7660999999999997e-05 2024-04-25T20:09:29.629 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.629 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.629 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:29.629 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129633+0000", 2024-04-25T20:09:29.629 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8649999999999998e-05 2024-04-25T20:09:29.629 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.629 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:29.629 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.629 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.630 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.630 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:135 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:29.630 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:09:29.630 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 11.449891571, 2024-04-25T20:09:29.630 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010898453000000001, 2024-04-25T20:09:29.630 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:29.630 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:29.630 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:29.630 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:29.630 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:29.630 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 135 2024-04-25T20:09:29.630 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.630 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:29.631 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.631 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:29.631 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:09:29.631 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.631 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.631 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.631 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:29.631 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:09:29.631 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.631 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.631 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.631 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:29.631 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121214+0000", 2024-04-25T20:09:29.632 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:29.632 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.632 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.632 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:29.632 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121218+0000", 2024-04-25T20:09:29.632 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6619999999999998e-06 2024-04-25T20:09:29.632 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.632 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.632 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:29.632 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121223+0000", 2024-04-25T20:09:29.632 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1070000000000004e-06 2024-04-25T20:09:29.633 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.633 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.633 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:29.633 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121227+0000", 2024-04-25T20:09:29.633 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2820000000000001e-06 2024-04-25T20:09:29.633 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.633 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.633 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:29.633 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123649+0000", 2024-04-25T20:09:29.633 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0024221970000000001 2024-04-25T20:09:29.633 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.634 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.634 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:29.634 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123723+0000", 2024-04-25T20:09:29.634 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.4386000000000002e-05 2024-04-25T20:09:29.634 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.634 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.634 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:29.634 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123844+0000", 2024-04-25T20:09:29.634 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000121249 2024-04-25T20:09:29.634 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.634 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.635 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:29.635 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125670+0000", 2024-04-25T20:09:29.635 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001825901 2024-04-25T20:09:29.635 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.635 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.635 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:29.635 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132043+0000", 2024-04-25T20:09:29.636 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0063724539999999996 2024-04-25T20:09:29.636 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.636 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.636 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:29.636 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132084+0000", 2024-04-25T20:09:29.636 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0963000000000001e-05 2024-04-25T20:09:29.636 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.636 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.636 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:29.637 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132114+0000", 2024-04-25T20:09:29.637 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9927e-05 2024-04-25T20:09:29.637 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.637 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:29.637 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.637 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.637 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.637 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:136 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:29.638 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:09:29.638 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 11.449876045, 2024-04-25T20:09:29.638 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010952513000000001, 2024-04-25T20:09:29.638 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:29.638 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:29.638 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:29.638 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:29.638 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:29.639 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 136 2024-04-25T20:09:29.639 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.639 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:29.639 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.639 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:29.639 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:09:29.639 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.640 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.640 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.640 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:29.640 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:09:29.640 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.640 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.641 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.641 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:29.641 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121230+0000", 2024-04-25T20:09:29.641 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:29.641 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.641 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.641 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:29.642 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121236+0000", 2024-04-25T20:09:29.642 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.7320000000000003e-06 2024-04-25T20:09:29.642 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.642 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.642 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:29.642 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121242+0000", 2024-04-25T20:09:29.642 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.7810000000000002e-06 2024-04-25T20:09:29.643 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.643 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.643 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:29.643 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121246+0000", 2024-04-25T20:09:29.643 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.639e-06 2024-04-25T20:09:29.643 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.643 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.643 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:29.643 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124150+0000", 2024-04-25T20:09:29.644 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.002903816 2024-04-25T20:09:29.644 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.644 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.644 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:29.644 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124195+0000", 2024-04-25T20:09:29.644 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.5042e-05 2024-04-25T20:09:29.644 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.645 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.645 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:29.645 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124266+0000", 2024-04-25T20:09:29.645 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0234999999999994e-05 2024-04-25T20:09:29.645 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.645 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.646 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:29.646 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125711+0000", 2024-04-25T20:09:29.646 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0014458800000000001 2024-04-25T20:09:29.646 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.646 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.646 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:29.647 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132120+0000", 2024-04-25T20:09:29.647 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0064084399999999996 2024-04-25T20:09:29.647 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.647 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.647 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:29.647 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132155+0000", 2024-04-25T20:09:29.647 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4938000000000003e-05 2024-04-25T20:09:29.647 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.647 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.647 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:29.648 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132183+0000", 2024-04-25T20:09:29.648 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8360999999999998e-05 2024-04-25T20:09:29.648 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.648 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:29.648 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.648 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.648 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.648 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:29.648 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:09:29.648 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 11.449826312999999, 2024-04-25T20:09:29.648 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.01097684, 2024-04-25T20:09:29.648 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:29.648 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:29.648 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:29.648 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:29.649 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:29.649 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 137 2024-04-25T20:09:29.649 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.649 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:29.649 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.649 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:29.649 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:09:29.649 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.649 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.649 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.649 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:29.649 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:09:29.649 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.649 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.649 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.650 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:29.650 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:09:29.650 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:29.650 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.650 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.650 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:29.650 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121328+0000", 2024-04-25T20:09:29.650 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.7908999999999998e-05 2024-04-25T20:09:29.650 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.650 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.650 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:29.650 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121337+0000", 2024-04-25T20:09:29.651 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.9369999999999993e-06 2024-04-25T20:09:29.651 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.651 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.651 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:29.651 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121343+0000", 2024-04-25T20:09:29.651 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.8089999999999998e-06 2024-04-25T20:09:29.651 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.651 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.651 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:29.651 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124656+0000", 2024-04-25T20:09:29.651 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0033133839999999999 2024-04-25T20:09:29.651 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.652 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.652 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:29.652 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124730+0000", 2024-04-25T20:09:29.652 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3565000000000005e-05 2024-04-25T20:09:29.652 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.652 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.652 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:29.652 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124846+0000", 2024-04-25T20:09:29.652 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000116416 2024-04-25T20:09:29.652 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.652 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.652 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:29.652 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126756+0000", 2024-04-25T20:09:29.652 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001909991 2024-04-25T20:09:29.652 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.653 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.653 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:29.653 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132190+0000", 2024-04-25T20:09:29.653 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0054343880000000001 2024-04-25T20:09:29.653 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.653 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.653 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:29.653 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132227+0000", 2024-04-25T20:09:29.653 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6875000000000002e-05 2024-04-25T20:09:29.653 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.653 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.653 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:29.653 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132257+0000", 2024-04-25T20:09:29.653 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.0046e-05 2024-04-25T20:09:29.654 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.654 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:29.654 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.654 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.654 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.654 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:29.654 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:09:29.654 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 11.449756355, 2024-04-25T20:09:29.654 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010978653999999999, 2024-04-25T20:09:29.654 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:29.655 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:29.655 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:29.655 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:29.655 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:29.655 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 138 2024-04-25T20:09:29.655 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.655 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:29.655 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.655 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:29.655 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:09:29.655 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.655 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.655 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.655 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:29.655 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:09:29.656 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.656 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.656 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.656 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:29.656 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:09:29.656 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:29.656 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.656 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.656 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:29.656 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121355+0000", 2024-04-25T20:09:29.656 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.9869999999999999e-06 2024-04-25T20:09:29.656 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.656 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.656 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:29.657 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121361+0000", 2024-04-25T20:09:29.657 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.4620000000000001e-06 2024-04-25T20:09:29.657 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.657 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.657 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:29.657 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121373+0000", 2024-04-25T20:09:29.657 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.2035000000000001e-05 2024-04-25T20:09:29.657 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.657 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.657 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:29.657 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125267+0000", 2024-04-25T20:09:29.657 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0038939230000000001 2024-04-25T20:09:29.657 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.657 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.657 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:29.658 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125309+0000", 2024-04-25T20:09:29.658 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1690999999999998e-05 2024-04-25T20:09:29.658 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.658 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.658 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:29.658 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125385+0000", 2024-04-25T20:09:29.658 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.6162999999999996e-05 2024-04-25T20:09:29.658 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.658 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.659 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:29.659 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126798+0000", 2024-04-25T20:09:29.659 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0014124680000000001 2024-04-25T20:09:29.659 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.659 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.659 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:29.659 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132264+0000", 2024-04-25T20:09:29.659 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0054661689999999999 2024-04-25T20:09:29.659 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.659 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.659 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:29.659 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132299+0000", 2024-04-25T20:09:29.660 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5272999999999999e-05 2024-04-25T20:09:29.660 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.660 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.660 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:29.660 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132329+0000", 2024-04-25T20:09:29.660 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.0080999999999999e-05 2024-04-25T20:09:29.660 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.660 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:29.660 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.660 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.660 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.661 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:29.661 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:09:29.661 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 11.44972578, 2024-04-25T20:09:29.661 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011015853000000001, 2024-04-25T20:09:29.661 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:29.661 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:29.661 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:29.661 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:29.661 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:29.661 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 139 2024-04-25T20:09:29.661 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.662 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:29.662 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.662 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:29.662 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:09:29.662 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.662 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.662 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.662 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:29.662 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:09:29.662 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.662 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.662 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.663 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:29.663 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121380+0000", 2024-04-25T20:09:29.663 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:29.663 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.663 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.663 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:29.663 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121385+0000", 2024-04-25T20:09:29.663 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.092e-06 2024-04-25T20:09:29.664 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.664 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.664 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:29.664 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121390+0000", 2024-04-25T20:09:29.664 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.5990000000000001e-06 2024-04-25T20:09:29.664 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.664 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.664 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:29.664 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121397+0000", 2024-04-25T20:09:29.664 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.6279999999999999e-06 2024-04-25T20:09:29.665 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.665 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.665 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:29.665 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125737+0000", 2024-04-25T20:09:29.665 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0043406010000000004 2024-04-25T20:09:29.665 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.665 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.665 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:29.665 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125781+0000", 2024-04-25T20:09:29.665 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.3958999999999997e-05 2024-04-25T20:09:29.665 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.665 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.665 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:29.665 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125849+0000", 2024-04-25T20:09:29.666 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8071999999999995e-05 2024-04-25T20:09:29.666 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.666 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.666 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:29.666 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.130908+0000", 2024-04-25T20:09:29.666 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0050586509999999999 2024-04-25T20:09:29.666 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.666 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.666 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:29.666 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132334+0000", 2024-04-25T20:09:29.666 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00142621 2024-04-25T20:09:29.666 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.666 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.667 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:29.667 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132369+0000", 2024-04-25T20:09:29.667 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4452000000000002e-05 2024-04-25T20:09:29.667 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.667 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.667 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:29.667 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132397+0000", 2024-04-25T20:09:29.667 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8065e-05 2024-04-25T20:09:29.667 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.667 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:29.667 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.667 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.668 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.668 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:29.668 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:09:29.668 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 11.449704735999999, 2024-04-25T20:09:29.668 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011069443, 2024-04-25T20:09:29.668 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:29.668 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:29.668 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:29.668 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:29.668 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:29.668 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 140 2024-04-25T20:09:29.668 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.668 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:29.669 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.669 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:29.669 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:09:29.669 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.669 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.669 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.669 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:29.669 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:09:29.669 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.669 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.669 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.669 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:29.669 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:09:29.669 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:29.670 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.670 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.670 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:29.670 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121456+0000", 2024-04-25T20:09:29.670 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.4469000000000003e-05 2024-04-25T20:09:29.670 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.670 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.670 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:29.670 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121462+0000", 2024-04-25T20:09:29.670 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.1890000000000004e-06 2024-04-25T20:09:29.671 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.671 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.671 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:29.671 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121466+0000", 2024-04-25T20:09:29.671 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3909999999999998e-06 2024-04-25T20:09:29.671 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.671 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.671 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:29.671 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126071+0000", 2024-04-25T20:09:29.671 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0046048979999999996 2024-04-25T20:09:29.672 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.672 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.672 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:29.672 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126141+0000", 2024-04-25T20:09:29.672 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0621e-05 2024-04-25T20:09:29.672 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.672 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.672 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:29.672 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126260+0000", 2024-04-25T20:09:29.672 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000118621 2024-04-25T20:09:29.672 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.672 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.673 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:29.673 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.131270+0000", 2024-04-25T20:09:29.673 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0050096949999999998 2024-04-25T20:09:29.673 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.673 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.673 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:29.673 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132403+0000", 2024-04-25T20:09:29.673 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011332779999999999 2024-04-25T20:09:29.673 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.673 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.673 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:29.673 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132443+0000", 2024-04-25T20:09:29.673 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0175000000000001e-05 2024-04-25T20:09:29.673 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.674 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.674 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:29.674 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132471+0000", 2024-04-25T20:09:29.674 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8399e-05 2024-04-25T20:09:29.674 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.674 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:29.674 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.674 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.674 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.674 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:29.674 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:09:29.675 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 11.449635950999999, 2024-04-25T20:09:29.675 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011071002, 2024-04-25T20:09:29.675 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:29.675 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:29.675 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:29.675 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:29.675 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:29.675 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 141 2024-04-25T20:09:29.675 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.675 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:29.675 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.675 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:29.675 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:09:29.675 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.675 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.676 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.676 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:29.676 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:09:29.676 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.676 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.676 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.676 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:29.676 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121470+0000", 2024-04-25T20:09:29.676 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:29.676 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.676 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.676 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:29.676 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121473+0000", 2024-04-25T20:09:29.677 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.01e-06 2024-04-25T20:09:29.677 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.677 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.677 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:29.677 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121478+0000", 2024-04-25T20:09:29.677 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.515e-06 2024-04-25T20:09:29.677 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.677 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.677 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:29.677 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121480+0000", 2024-04-25T20:09:29.677 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5749999999999999e-06 2024-04-25T20:09:29.677 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.677 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.677 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:29.677 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126813+0000", 2024-04-25T20:09:29.678 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0053320399999999997 2024-04-25T20:09:29.678 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.678 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.678 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:29.678 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126854+0000", 2024-04-25T20:09:29.678 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1582999999999998e-05 2024-04-25T20:09:29.678 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.678 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.678 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:29.678 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126923+0000", 2024-04-25T20:09:29.678 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8544e-05 2024-04-25T20:09:29.678 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.678 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.679 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:29.679 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.131573+0000", 2024-04-25T20:09:29.679 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0046504659999999998 2024-04-25T20:09:29.679 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.679 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.679 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:29.679 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132478+0000", 2024-04-25T20:09:29.679 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00090442400000000003 2024-04-25T20:09:29.679 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.679 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.679 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:29.679 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132513+0000", 2024-04-25T20:09:29.679 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5204999999999999e-05 2024-04-25T20:09:29.679 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.680 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.680 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:29.680 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132542+0000", 2024-04-25T20:09:29.680 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.904e-05 2024-04-25T20:09:29.680 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.680 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:29.680 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.680 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.680 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.680 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:156 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:29.680 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:09:29.680 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 10.951811656, 2024-04-25T20:09:29.680 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090072280000000008, 2024-04-25T20:09:29.680 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:29.681 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:29.681 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:29.681 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:29.681 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:29.681 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 156 2024-04-25T20:09:29.681 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.681 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:29.681 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.681 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:29.681 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:09:29.681 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.681 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.681 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.681 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:29.682 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:09:29.682 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.682 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.682 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.682 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:29.682 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:09:29.682 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:29.682 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.682 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.682 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:29.682 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619297+0000", 2024-04-25T20:09:29.682 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2979999999999999e-06 2024-04-25T20:09:29.682 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.682 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.683 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:29.683 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619300+0000", 2024-04-25T20:09:29.683 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.371e-06 2024-04-25T20:09:29.683 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.683 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.683 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:29.683 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619304+0000", 2024-04-25T20:09:29.683 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.304e-06 2024-04-25T20:09:29.683 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.683 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.683 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:29.683 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.622910+0000", 2024-04-25T20:09:29.683 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0036065049999999999 2024-04-25T20:09:29.683 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.684 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.684 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:29.684 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.622990+0000", 2024-04-25T20:09:29.684 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.9722999999999995e-05 2024-04-25T20:09:29.684 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.684 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.684 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:29.684 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623128+0000", 2024-04-25T20:09:29.684 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00013818399999999999 2024-04-25T20:09:29.684 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.684 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.684 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:29.684 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625155+0000", 2024-04-25T20:09:29.684 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0020267979999999998 2024-04-25T20:09:29.684 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.685 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.685 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:29.685 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628225+0000", 2024-04-25T20:09:29.685 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0030705680000000001 2024-04-25T20:09:29.685 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.685 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.685 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:29.685 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628268+0000", 2024-04-25T20:09:29.685 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.2404000000000003e-05 2024-04-25T20:09:29.685 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.685 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.685 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:29.685 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628302+0000", 2024-04-25T20:09:29.685 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4439e-05 2024-04-25T20:09:29.686 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.686 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:29.686 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.686 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.686 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.686 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:157 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:29.686 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:09:29.686 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 10.951799556999999, 2024-04-25T20:09:29.686 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090534889999999996, 2024-04-25T20:09:29.686 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:29.686 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:29.686 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:29.686 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:29.687 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:29.687 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 157 2024-04-25T20:09:29.687 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.687 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:29.687 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.687 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:29.687 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:09:29.687 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.687 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.687 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.687 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:29.687 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:09:29.687 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.687 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.687 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.688 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:29.688 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:09:29.688 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:29.688 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.688 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.688 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:29.688 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619309+0000", 2024-04-25T20:09:29.688 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.841e-06 2024-04-25T20:09:29.688 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.688 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.688 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:29.688 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619312+0000", 2024-04-25T20:09:29.688 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.084e-06 2024-04-25T20:09:29.688 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.688 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.689 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:29.689 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619314+0000", 2024-04-25T20:09:29.689 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.3860000000000001e-06 2024-04-25T20:09:29.689 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.689 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.689 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:29.689 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623718+0000", 2024-04-25T20:09:29.689 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0044038050000000002 2024-04-25T20:09:29.689 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.689 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.689 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:29.690 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623791+0000", 2024-04-25T20:09:29.690 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3181000000000003e-05 2024-04-25T20:09:29.690 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.690 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.690 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:29.690 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623922+0000", 2024-04-25T20:09:29.690 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00013117000000000001 2024-04-25T20:09:29.690 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.690 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.690 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:29.690 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625187+0000", 2024-04-25T20:09:29.691 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001264961 2024-04-25T20:09:29.691 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.691 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.691 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:29.691 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628308+0000", 2024-04-25T20:09:29.691 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.003120739 2024-04-25T20:09:29.691 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.691 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.691 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:29.691 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628331+0000", 2024-04-25T20:09:29.691 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2802999999999998e-05 2024-04-25T20:09:29.691 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.691 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.691 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:29.692 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628361+0000", 2024-04-25T20:09:29.692 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9760999999999998e-05 2024-04-25T20:09:29.692 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.692 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:29.692 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.692 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.692 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.692 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:158 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:29.692 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:09:29.692 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 10.95178911, 2024-04-25T20:09:29.692 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0091076409999999997, 2024-04-25T20:09:29.692 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:29.692 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:29.692 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:29.692 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:29.693 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:29.693 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 158 2024-04-25T20:09:29.693 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.693 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:29.693 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.693 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:29.693 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:09:29.693 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.693 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.693 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.693 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:29.693 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:09:29.693 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.693 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.693 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.694 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:29.694 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619317+0000", 2024-04-25T20:09:29.694 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967296 2024-04-25T20:09:29.694 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.694 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.694 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:29.694 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619364+0000", 2024-04-25T20:09:29.694 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.6572000000000001e-05 2024-04-25T20:09:29.694 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.694 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.694 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:29.694 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619371+0000", 2024-04-25T20:09:29.694 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3860000000000001e-06 2024-04-25T20:09:29.695 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.695 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.695 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:29.695 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619382+0000", 2024-04-25T20:09:29.695 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0173999999999999e-05 2024-04-25T20:09:29.695 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.695 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.695 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:29.695 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624216+0000", 2024-04-25T20:09:29.695 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048340350000000004 2024-04-25T20:09:29.695 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.695 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.695 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:29.695 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624260+0000", 2024-04-25T20:09:29.696 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4558999999999998e-05 2024-04-25T20:09:29.696 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.696 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.696 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:29.696 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624333+0000", 2024-04-25T20:09:29.696 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2744999999999996e-05 2024-04-25T20:09:29.696 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.696 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.696 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:29.696 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625465+0000", 2024-04-25T20:09:29.696 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001132454 2024-04-25T20:09:29.696 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.696 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.696 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:29.696 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628365+0000", 2024-04-25T20:09:29.697 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0028994950000000002 2024-04-25T20:09:29.697 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.697 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.697 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:29.697 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628386+0000", 2024-04-25T20:09:29.697 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.1367000000000001e-05 2024-04-25T20:09:29.697 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.697 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.697 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:29.697 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628425+0000", 2024-04-25T20:09:29.697 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.9087e-05 2024-04-25T20:09:29.697 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.697 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:29.697 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.698 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.698 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.698 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:159 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:29.698 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:09:29.698 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 10.951720457, 2024-04-25T20:09:29.698 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090948999999999995, 2024-04-25T20:09:29.698 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:29.698 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:29.698 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:29.698 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:29.698 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:29.698 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 159 2024-04-25T20:09:29.698 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.698 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:29.699 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.699 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:29.699 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:09:29.699 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.699 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.699 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.699 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:29.699 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:09:29.699 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:29.699 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.699 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.699 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:29.699 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:09:29.699 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:29.699 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.700 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.700 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:29.700 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619389+0000", 2024-04-25T20:09:29.700 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.734e-06 2024-04-25T20:09:29.700 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.700 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.700 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:29.700 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619392+0000", 2024-04-25T20:09:29.700 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6219999999999998e-06 2024-04-25T20:09:29.700 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.700 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.700 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:29.700 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619403+0000", 2024-04-25T20:09:29.700 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0987e-05 2024-04-25T20:09:29.700 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.701 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.701 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:29.701 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624667+0000", 2024-04-25T20:09:29.701 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0052633970000000004 2024-04-25T20:09:29.701 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.701 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.701 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:29.701 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624741+0000", 2024-04-25T20:09:29.701 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.4583000000000001e-05 2024-04-25T20:09:29.701 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.701 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.701 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:29.701 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624857+0000", 2024-04-25T20:09:29.701 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000116118 2024-04-25T20:09:29.702 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.702 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.702 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:29.702 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.626013+0000", 2024-04-25T20:09:29.702 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001155473 2024-04-25T20:09:29.702 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.702 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.702 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:29.702 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628432+0000", 2024-04-25T20:09:29.702 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0024189379999999998 2024-04-25T20:09:29.702 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.702 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.702 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:29.702 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628454+0000", 2024-04-25T20:09:29.703 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2059999999999999e-05 2024-04-25T20:09:29.703 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:29.703 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:29.703 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:29.703 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628481+0000", 2024-04-25T20:09:29.703 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.7300999999999998e-05 2024-04-25T20:09:29.703 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.703 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:29.703 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.703 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:29.703 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:29.703 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:29.705 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:29.705+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:29.705 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T20:09:29.805 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:29.805+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:29.812 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:29.812 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:29.812 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:09:29.812 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:29.821 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T20:09:29.906 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:29.905+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:29.926 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:29.926 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:29.926 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:09:29.926 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:09:29.926 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:29.934 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T20:09:30.018 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:30.017+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:30.068 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is set 2024-04-25T20:09:30.085 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:30.085 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:09:30.085 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:09:30.085 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:09:30.085 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:30.094 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T20:09:30.108 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:30.108+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:30.209 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:30.208+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:30.241 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:30.241 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:30.241 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:09:30.241 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:30.250 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T20:09:30.311 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:30.310+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:30.396 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:30.396 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:30.397 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:09:30.397 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:09:30.397 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:30.406 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T20:09:30.450 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:30.449+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:30.511 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:30.512+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:30.513 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:30.513 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:09:30.513 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:09:30.513 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-04-25T20:09:30.513 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.513 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:30 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'16, mlcod=16'14)", 2024-04-25T20:09:30.513 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:09:30.513 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.344919249, 2024-04-25T20:09:30.513 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0054087670000000001, 2024-04-25T20:09:30.513 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:30.514 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:30.514 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:30.514 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.514 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:30.514 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:09:30.514 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.514 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.514 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.514 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:30.514 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:09:30.514 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.514 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.514 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.514 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:30.514 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167142+0000", 2024-04-25T20:09:30.515 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999976 2024-04-25T20:09:30.515 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.515 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.515 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:30.515 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167162+0000", 2024-04-25T20:09:30.515 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0109e-05 2024-04-25T20:09:30.515 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.515 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.515 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:30.515 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167188+0000", 2024-04-25T20:09:30.515 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6146e-05 2024-04-25T20:09:30.515 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.515 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.515 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:30.515 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167199+0000", 2024-04-25T20:09:30.516 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0767e-05 2024-04-25T20:09:30.516 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.516 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.516 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:30.516 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167256+0000", 2024-04-25T20:09:30.516 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7392000000000002e-05 2024-04-25T20:09:30.516 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.516 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.516 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:30.516 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167283+0000", 2024-04-25T20:09:30.516 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6577e-05 2024-04-25T20:09:30.516 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.516 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.517 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:30.517 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.172510+0000", 2024-04-25T20:09:30.517 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0052276759999999997 2024-04-25T20:09:30.517 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.517 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.517 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:30.517 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.172553+0000", 2024-04-25T20:09:30.517 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2444999999999998e-05 2024-04-25T20:09:30.517 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.517 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:30.517 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.517 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.517 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.517 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:31 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-04-25T20:09:30.517 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:09:30.518 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.344303171, 2024-04-25T20:09:30.518 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0055055479999999999, 2024-04-25T20:09:30.518 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:30.518 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:30.518 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:30.518 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.518 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:30.518 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:09:30.518 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.518 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.518 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.518 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:30.518 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:09:30.519 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.519 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.519 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.519 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:30.519 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:09:30.519 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:30.519 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.519 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.519 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:30.519 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167766+0000", 2024-04-25T20:09:30.519 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.122e-06 2024-04-25T20:09:30.519 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.519 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.519 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:30.519 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167776+0000", 2024-04-25T20:09:30.520 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0515e-05 2024-04-25T20:09:30.520 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.520 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.520 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:30.520 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167782+0000", 2024-04-25T20:09:30.520 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1149999999999999e-06 2024-04-25T20:09:30.520 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.520 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.520 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:30.520 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167837+0000", 2024-04-25T20:09:30.520 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4763999999999999e-05 2024-04-25T20:09:30.520 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.520 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.520 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:30.521 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167855+0000", 2024-04-25T20:09:30.521 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7589e-05 2024-04-25T20:09:30.521 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.521 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.521 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:30.521 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173221+0000", 2024-04-25T20:09:30.521 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.005365903 2024-04-25T20:09:30.521 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.521 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.521 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:30.521 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173266+0000", 2024-04-25T20:09:30.521 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5059999999999999e-05 2024-04-25T20:09:30.521 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.521 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:30.522 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.522 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.522 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.522 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:32 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-04-25T20:09:30.522 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:09:30.522 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.343927986000001, 2024-04-25T20:09:30.522 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0051558239999999998, 2024-04-25T20:09:30.522 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:30.522 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:30.522 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:30.522 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.522 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:30.522 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:09:30.522 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.523 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.523 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.523 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:30.523 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:09:30.523 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.523 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.523 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.523 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:30.523 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:09:30.523 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:30.523 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.523 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.523 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:30.523 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168173+0000", 2024-04-25T20:09:30.523 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7663999999999997e-05 2024-04-25T20:09:30.524 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.524 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.524 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:30.524 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168182+0000", 2024-04-25T20:09:30.524 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0650000000000005e-06 2024-04-25T20:09:30.524 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.524 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.524 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:30.524 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168188+0000", 2024-04-25T20:09:30.524 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0589999999999999e-06 2024-04-25T20:09:30.524 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.524 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.524 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:30.524 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168239+0000", 2024-04-25T20:09:30.524 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1467e-05 2024-04-25T20:09:30.525 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.525 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.525 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:30.525 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168260+0000", 2024-04-25T20:09:30.525 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0531e-05 2024-04-25T20:09:30.525 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.525 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.525 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:30.525 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173268+0000", 2024-04-25T20:09:30.525 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0050087980000000001 2024-04-25T20:09:30.525 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.525 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.525 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:30.525 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173291+0000", 2024-04-25T20:09:30.526 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.268e-05 2024-04-25T20:09:30.526 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.526 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:30.526 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.526 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.526 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.526 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:33 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'19, mlcod=16'14)", 2024-04-25T20:09:30.526 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:09:30.526 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.343417566999999, 2024-04-25T20:09:30.526 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046697049999999997, 2024-04-25T20:09:30.526 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:30.526 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:30.526 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:30.526 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.527 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:30.527 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:09:30.527 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.527 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.527 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.527 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:30.527 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:09:30.527 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.527 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.527 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.527 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:30.527 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168645+0000", 2024-04-25T20:09:30.527 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:30.527 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.528 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.528 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:30.528 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168651+0000", 2024-04-25T20:09:30.528 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1079999999999998e-06 2024-04-25T20:09:30.528 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.528 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.528 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:30.528 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168660+0000", 2024-04-25T20:09:30.528 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0010000000000007e-06 2024-04-25T20:09:30.528 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.528 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.528 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:30.528 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168666+0000", 2024-04-25T20:09:30.528 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7780000000000004e-06 2024-04-25T20:09:30.529 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.529 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.529 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:30.529 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168701+0000", 2024-04-25T20:09:30.529 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4490999999999999e-05 2024-04-25T20:09:30.529 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.529 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.529 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:30.529 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168729+0000", 2024-04-25T20:09:30.529 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8476999999999999e-05 2024-04-25T20:09:30.529 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.529 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.529 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:30.529 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173294+0000", 2024-04-25T20:09:30.529 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0045650730000000002 2024-04-25T20:09:30.530 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.530 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.530 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:30.530 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173315+0000", 2024-04-25T20:09:30.530 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1245000000000001e-05 2024-04-25T20:09:30.530 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.530 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:30.530 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.530 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.530 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.530 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:34 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'20, mlcod=16'14)", 2024-04-25T20:09:30.530 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:09:30.530 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.342918638, 2024-04-25T20:09:30.530 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041943309999999999, 2024-04-25T20:09:30.531 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:30.531 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:30.531 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:30.531 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.531 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:30.531 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:09:30.531 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.531 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.531 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.531 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:30.531 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:09:30.531 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.531 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.531 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.531 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:30.531 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169144+0000", 2024-04-25T20:09:30.532 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:30.532 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.532 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.532 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:30.532 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169150+0000", 2024-04-25T20:09:30.532 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9150000000000001e-06 2024-04-25T20:09:30.532 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.532 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.532 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:30.532 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169159+0000", 2024-04-25T20:09:30.532 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8899999999999996e-06 2024-04-25T20:09:30.532 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.532 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.532 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:30.532 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169169+0000", 2024-04-25T20:09:30.532 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0441e-05 2024-04-25T20:09:30.533 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.533 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.533 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:30.533 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169206+0000", 2024-04-25T20:09:30.533 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6816000000000001e-05 2024-04-25T20:09:30.533 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.533 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.533 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:30.533 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169234+0000", 2024-04-25T20:09:30.533 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8008999999999999e-05 2024-04-25T20:09:30.533 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.533 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.533 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:30.533 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173317+0000", 2024-04-25T20:09:30.533 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0040830160000000001 2024-04-25T20:09:30.534 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.534 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.534 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:30.534 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173339+0000", 2024-04-25T20:09:30.534 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1897000000000001e-05 2024-04-25T20:09:30.534 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.534 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:30.534 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.534 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.534 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.534 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:35 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'14)", 2024-04-25T20:09:30.534 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:09:30.534 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.342426318999999, 2024-04-25T20:09:30.534 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037236529999999999, 2024-04-25T20:09:30.534 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:30.535 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:30.535 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:30.535 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.535 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:30.535 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:09:30.535 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.535 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.535 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.535 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:30.535 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:09:30.535 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.535 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.535 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.535 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:30.535 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169636+0000", 2024-04-25T20:09:30.536 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:30.536 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.536 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.536 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:30.536 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169678+0000", 2024-04-25T20:09:30.536 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1752000000000003e-05 2024-04-25T20:09:30.536 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.536 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.536 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:30.536 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169688+0000", 2024-04-25T20:09:30.536 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0098000000000001e-05 2024-04-25T20:09:30.536 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.536 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.536 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:30.536 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169694+0000", 2024-04-25T20:09:30.536 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2349999999999996e-06 2024-04-25T20:09:30.537 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.537 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.537 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:30.537 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169733+0000", 2024-04-25T20:09:30.537 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8481e-05 2024-04-25T20:09:30.537 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.537 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.537 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:30.537 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169750+0000", 2024-04-25T20:09:30.537 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7509000000000001e-05 2024-04-25T20:09:30.537 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.537 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.537 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:30.537 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173340+0000", 2024-04-25T20:09:30.537 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0035898449999999999 2024-04-25T20:09:30.538 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.538 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.538 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:30.538 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173361+0000", 2024-04-25T20:09:30.538 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.039e-05 2024-04-25T20:09:30.538 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.538 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:30.538 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.538 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.538 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.538 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:36 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'22, mlcod=16'14)", 2024-04-25T20:09:30.538 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:09:30.538 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.341926258999999, 2024-04-25T20:09:30.538 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032472120000000002, 2024-04-25T20:09:30.539 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:30.539 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:30.539 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:30.539 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.539 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:30.539 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:09:30.539 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.539 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.539 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.539 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:30.539 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:09:30.539 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.539 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.539 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.539 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:30.539 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:09:30.539 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:30.540 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.540 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.540 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:30.540 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170152+0000", 2024-04-25T20:09:30.540 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5798e-05 2024-04-25T20:09:30.540 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.540 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.540 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:30.540 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170161+0000", 2024-04-25T20:09:30.540 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.064e-06 2024-04-25T20:09:30.540 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.540 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.540 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:30.540 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170167+0000", 2024-04-25T20:09:30.540 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8390000000000003e-06 2024-04-25T20:09:30.541 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.541 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.541 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:30.541 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170202+0000", 2024-04-25T20:09:30.541 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.5120999999999998e-05 2024-04-25T20:09:30.541 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.541 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.541 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:30.541 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170219+0000", 2024-04-25T20:09:30.541 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6929999999999999e-05 2024-04-25T20:09:30.541 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.541 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.541 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:30.541 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173362+0000", 2024-04-25T20:09:30.541 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0031427619999999999 2024-04-25T20:09:30.542 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.542 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.542 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:30.542 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173384+0000", 2024-04-25T20:09:30.542 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2237000000000001e-05 2024-04-25T20:09:30.542 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.542 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:30.542 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.542 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.542 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.542 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:40 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'26, mlcod=16'24)", 2024-04-25T20:09:30.542 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:09:30.542 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.334888347, 2024-04-25T20:09:30.542 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028727330000000001, 2024-04-25T20:09:30.543 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:30.543 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:30.543 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:30.543 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.543 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:30.543 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:09:30.543 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.543 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.543 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.543 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:30.543 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:09:30.543 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.543 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.543 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.543 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:30.544 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177173+0000", 2024-04-25T20:09:30.544 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999976 2024-04-25T20:09:30.544 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.544 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.544 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:30.544 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177186+0000", 2024-04-25T20:09:30.544 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3533000000000001e-05 2024-04-25T20:09:30.544 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.544 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.544 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:30.544 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177209+0000", 2024-04-25T20:09:30.544 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3280000000000001e-05 2024-04-25T20:09:30.544 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.544 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.545 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:30.545 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177220+0000", 2024-04-25T20:09:30.545 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0465000000000001e-05 2024-04-25T20:09:30.545 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.545 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.545 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:30.545 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177264+0000", 2024-04-25T20:09:30.545 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4413999999999999e-05 2024-04-25T20:09:30.545 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.545 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.545 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:30.545 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177289+0000", 2024-04-25T20:09:30.545 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4783999999999999e-05 2024-04-25T20:09:30.545 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.546 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.546 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:30.546 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.179990+0000", 2024-04-25T20:09:30.546 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027012939999999999 2024-04-25T20:09:30.546 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.546 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.546 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:30.546 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.180048+0000", 2024-04-25T20:09:30.546 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7299000000000001e-05 2024-04-25T20:09:30.546 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.546 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:30.546 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.546 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.546 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.547 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:67 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'41, mlcod=16'39)", 2024-04-25T20:09:30.547 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:09:30.547 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.300126855, 2024-04-25T20:09:30.547 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064192549999999996, 2024-04-25T20:09:30.547 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:30.547 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:30.547 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:30.547 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.547 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:30.547 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:09:30.547 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.547 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.547 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.547 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:30.547 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:09:30.548 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.548 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.548 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.548 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:30.548 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211935+0000", 2024-04-25T20:09:30.548 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999981 2024-04-25T20:09:30.548 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.548 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.548 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:30.548 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211948+0000", 2024-04-25T20:09:30.548 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3531e-05 2024-04-25T20:09:30.548 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.548 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.548 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:30.549 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211967+0000", 2024-04-25T20:09:30.549 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8326999999999999e-05 2024-04-25T20:09:30.549 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.549 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.549 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:30.549 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211975+0000", 2024-04-25T20:09:30.549 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8589999999999994e-06 2024-04-25T20:09:30.549 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.549 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.549 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:30.549 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212020+0000", 2024-04-25T20:09:30.549 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4626999999999998e-05 2024-04-25T20:09:30.549 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.549 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.549 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:30.550 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212045+0000", 2024-04-25T20:09:30.550 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4508999999999999e-05 2024-04-25T20:09:30.550 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.550 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.550 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:30.550 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218316+0000", 2024-04-25T20:09:30.550 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0062711640000000001 2024-04-25T20:09:30.550 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.550 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.550 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:30.550 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218356+0000", 2024-04-25T20:09:30.550 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9993000000000001e-05 2024-04-25T20:09:30.550 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.550 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:30.551 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.551 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.551 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.551 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:68 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-04-25T20:09:30.551 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:09:30.551 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.299677840999999, 2024-04-25T20:09:30.551 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011231365, 2024-04-25T20:09:30.551 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:30.551 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:30.551 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:30.551 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.551 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:30.551 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:09:30.551 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.552 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.552 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.552 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:30.552 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:09:30.552 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.552 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.552 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.552 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:30.552 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:09:30.552 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:30.552 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.552 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.552 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:30.552 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212392+0000", 2024-04-25T20:09:30.552 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.79e-06 2024-04-25T20:09:30.553 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.553 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.553 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:30.553 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212406+0000", 2024-04-25T20:09:30.553 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3577000000000001e-05 2024-04-25T20:09:30.553 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.553 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.553 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:30.553 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212414+0000", 2024-04-25T20:09:30.553 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6510000000000001e-06 2024-04-25T20:09:30.553 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.553 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.553 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:30.553 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217114+0000", 2024-04-25T20:09:30.554 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047005379999999998 2024-04-25T20:09:30.554 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.554 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.554 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:30.554 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217141+0000", 2024-04-25T20:09:30.554 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6984000000000001e-05 2024-04-25T20:09:30.554 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.554 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.554 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:30.554 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223563+0000", 2024-04-25T20:09:30.554 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064216999999999998 2024-04-25T20:09:30.554 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.554 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.554 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:30.555 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223617+0000", 2024-04-25T20:09:30.555 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4011000000000002e-05 2024-04-25T20:09:30.555 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.555 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:30.555 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.555 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.555 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.555 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:69 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-04-25T20:09:30.555 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:09:30.555 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.299145651, 2024-04-25T20:09:30.555 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.010730094000000001, 2024-04-25T20:09:30.555 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:30.555 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:30.555 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:30.556 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.556 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:30.556 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:09:30.556 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.556 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.556 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.556 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:30.556 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:09:30.556 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.556 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.556 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.556 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:30.556 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212917+0000", 2024-04-25T20:09:30.556 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:30.557 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.557 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.557 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:30.557 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212957+0000", 2024-04-25T20:09:30.557 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0692000000000003e-05 2024-04-25T20:09:30.557 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.557 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.557 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:30.557 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212973+0000", 2024-04-25T20:09:30.557 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.575e-05 2024-04-25T20:09:30.557 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.557 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.557 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:30.557 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212980+0000", 2024-04-25T20:09:30.557 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1679999999999998e-06 2024-04-25T20:09:30.558 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.558 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.558 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:30.558 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217449+0000", 2024-04-25T20:09:30.558 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.004468367 2024-04-25T20:09:30.558 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.558 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.558 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:30.558 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217470+0000", 2024-04-25T20:09:30.558 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0749000000000001e-05 2024-04-25T20:09:30.558 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.558 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.558 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:30.558 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223621+0000", 2024-04-25T20:09:30.559 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0061512880000000004 2024-04-25T20:09:30.559 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.559 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.559 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:30.559 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223648+0000", 2024-04-25T20:09:30.559 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6961000000000002e-05 2024-04-25T20:09:30.559 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.559 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:30.559 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.559 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.559 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.559 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:70 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'44, mlcod=16'39)", 2024-04-25T20:09:30.559 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:09:30.559 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 13.298386923000001, 2024-04-25T20:09:30.559 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.013553358, 2024-04-25T20:09:30.560 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:30.560 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:30.560 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:30.560 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.560 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:30.560 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:09:30.560 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.560 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.560 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.560 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:30.560 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:09:30.560 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.560 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.560 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.561 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:30.561 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:09:30.561 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:30.561 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.561 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.561 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:30.561 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213682+0000", 2024-04-25T20:09:30.561 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3439999999999997e-06 2024-04-25T20:09:30.561 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.561 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.561 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:30.561 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213693+0000", 2024-04-25T20:09:30.561 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1049999999999999e-05 2024-04-25T20:09:30.561 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.561 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.562 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:30.562 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213699+0000", 2024-04-25T20:09:30.562 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9399999999999999e-06 2024-04-25T20:09:30.562 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.562 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.562 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:30.562 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218086+0000", 2024-04-25T20:09:30.562 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0043871240000000001 2024-04-25T20:09:30.562 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.562 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.562 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:30.562 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218106+0000", 2024-04-25T20:09:30.562 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.997e-05 2024-04-25T20:09:30.562 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.563 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.563 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:30.563 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.227186+0000", 2024-04-25T20:09:30.563 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.009079574 2024-04-25T20:09:30.563 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.563 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.563 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:30.563 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.227230+0000", 2024-04-25T20:09:30.563 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4147999999999998e-05 2024-04-25T20:09:30.563 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.563 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:30.563 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.563 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.563 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.563 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:106 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-04-25T20:09:30.564 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:09:30.564 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 12.927247823, 2024-04-25T20:09:30.564 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002936756, 2024-04-25T20:09:30.564 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:30.564 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:30.564 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:30.564 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.564 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:30.564 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:09:30.564 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.564 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.564 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.564 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:30.564 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:09:30.564 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.565 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.565 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.565 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:30.565 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584815+0000", 2024-04-25T20:09:30.565 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:30.565 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.565 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.565 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:30.565 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584860+0000", 2024-04-25T20:09:30.565 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5161999999999999e-05 2024-04-25T20:09:30.565 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.565 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.565 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:30.565 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584874+0000", 2024-04-25T20:09:30.565 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4464000000000001e-05 2024-04-25T20:09:30.566 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.566 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.566 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:30.566 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584881+0000", 2024-04-25T20:09:30.566 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.409e-06 2024-04-25T20:09:30.566 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.566 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.566 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:30.566 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584915+0000", 2024-04-25T20:09:30.566 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4007000000000002e-05 2024-04-25T20:09:30.566 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.566 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.566 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:30.566 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584933+0000", 2024-04-25T20:09:30.567 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7737999999999999e-05 2024-04-25T20:09:30.567 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.567 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.567 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:30.567 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587717+0000", 2024-04-25T20:09:30.567 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002784711 2024-04-25T20:09:30.567 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.567 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.567 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:30.567 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587752+0000", 2024-04-25T20:09:30.567 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4915e-05 2024-04-25T20:09:30.567 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.567 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:30.567 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.568 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.568 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.568 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:107 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'69, mlcod=16'64)", 2024-04-25T20:09:30.568 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:09:30.568 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 12.927172520999999, 2024-04-25T20:09:30.568 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028906270000000002, 2024-04-25T20:09:30.568 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:30.568 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:30.568 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:30.568 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.568 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:30.568 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:09:30.568 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.568 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.569 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.569 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:30.569 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:09:30.569 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.569 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.569 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.569 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:30.569 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584890+0000", 2024-04-25T20:09:30.569 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:30.569 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.569 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.569 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:30.569 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584895+0000", 2024-04-25T20:09:30.569 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2069999999999997e-06 2024-04-25T20:09:30.570 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.570 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.570 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:30.570 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584906+0000", 2024-04-25T20:09:30.570 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.117e-05 2024-04-25T20:09:30.570 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.570 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.570 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:30.570 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584910+0000", 2024-04-25T20:09:30.570 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4190000000000004e-06 2024-04-25T20:09:30.570 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.571 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.571 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:30.571 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585310+0000", 2024-04-25T20:09:30.571 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00040002700000000001 2024-04-25T20:09:30.571 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.571 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.571 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:30.571 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585329+0000", 2024-04-25T20:09:30.571 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8838000000000002e-05 2024-04-25T20:09:30.571 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.571 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.571 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:30.571 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587754+0000", 2024-04-25T20:09:30.571 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024252689999999999 2024-04-25T20:09:30.572 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.572 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.572 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:30.572 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587781+0000", 2024-04-25T20:09:30.572 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7206999999999999e-05 2024-04-25T20:09:30.572 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.572 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:30.572 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.572 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.572 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.573 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:108 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'70, mlcod=16'64)", 2024-04-25T20:09:30.573 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:09:30.573 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 12.92714737, 2024-04-25T20:09:30.573 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028924879999999999, 2024-04-25T20:09:30.573 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:30.573 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:30.573 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:30.573 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.573 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:30.573 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:09:30.573 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.573 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.573 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.573 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:30.574 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:09:30.574 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.574 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.574 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.574 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:30.574 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:09:30.574 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:30.574 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.574 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.574 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:30.574 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584919+0000", 2024-04-25T20:09:30.574 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.557e-06 2024-04-25T20:09:30.574 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.575 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.575 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:30.575 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584925+0000", 2024-04-25T20:09:30.575 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8619999999999999e-06 2024-04-25T20:09:30.575 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.575 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.575 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:30.575 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584931+0000", 2024-04-25T20:09:30.575 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2539999999999998e-06 2024-04-25T20:09:30.575 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.575 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.575 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:30.575 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585550+0000", 2024-04-25T20:09:30.575 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00061821800000000002 2024-04-25T20:09:30.575 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.575 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.576 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:30.576 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585567+0000", 2024-04-25T20:09:30.576 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7499e-05 2024-04-25T20:09:30.576 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.576 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.576 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:30.576 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587783+0000", 2024-04-25T20:09:30.576 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0022156200000000002 2024-04-25T20:09:30.576 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.576 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.576 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:30.576 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587808+0000", 2024-04-25T20:09:30.576 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5729e-05 2024-04-25T20:09:30.576 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.577 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:30.577 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.577 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.577 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.577 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:112 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'74, mlcod=16'66)", 2024-04-25T20:09:30.577 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:09:30.577 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 12.924989748, 2024-04-25T20:09:30.577 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00318724, 2024-04-25T20:09:30.577 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:30.577 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:30.577 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:30.577 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.577 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:30.577 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:09:30.578 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.578 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.578 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.578 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:30.578 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:09:30.578 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.578 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.578 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.578 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:30.578 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587073+0000", 2024-04-25T20:09:30.578 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:30.578 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.578 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.578 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:30.578 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587136+0000", 2024-04-25T20:09:30.579 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2928999999999993e-05 2024-04-25T20:09:30.579 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.579 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.579 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:30.579 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587147+0000", 2024-04-25T20:09:30.579 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1182999999999999e-05 2024-04-25T20:09:30.579 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.579 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.579 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:30.579 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587153+0000", 2024-04-25T20:09:30.579 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9059999999999999e-06 2024-04-25T20:09:30.579 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.579 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.579 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:30.579 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587306+0000", 2024-04-25T20:09:30.580 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000152848 2024-04-25T20:09:30.580 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.580 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.580 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:30.580 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587330+0000", 2024-04-25T20:09:30.580 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4304e-05 2024-04-25T20:09:30.580 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.580 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.580 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:30.580 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.590230+0000", 2024-04-25T20:09:30.580 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029004629999999998 2024-04-25T20:09:30.580 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.580 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.580 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:30.580 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.590261+0000", 2024-04-25T20:09:30.580 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0323000000000001e-05 2024-04-25T20:09:30.581 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.581 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:30.581 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.581 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.581 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.581 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:118 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'80, mlcod=16'78)", 2024-04-25T20:09:30.581 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:09:30.581 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 12.920203169000001, 2024-04-25T20:09:30.581 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029618959999999999, 2024-04-25T20:09:30.581 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:30.581 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:30.581 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:30.581 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.581 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:30.581 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:09:30.582 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.582 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.582 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.582 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:30.582 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:09:30.582 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.582 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.582 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.582 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:30.582 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591859+0000", 2024-04-25T20:09:30.582 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:30.582 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.582 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.582 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:30.583 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591918+0000", 2024-04-25T20:09:30.583 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8842999999999999e-05 2024-04-25T20:09:30.583 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.583 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.583 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:30.583 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591933+0000", 2024-04-25T20:09:30.583 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4600000000000001e-05 2024-04-25T20:09:30.583 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.583 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.583 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:30.583 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591939+0000", 2024-04-25T20:09:30.583 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4559999999999997e-06 2024-04-25T20:09:30.583 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.583 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.584 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:30.584 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591990+0000", 2024-04-25T20:09:30.584 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0318999999999997e-05 2024-04-25T20:09:30.584 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.584 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.584 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:30.584 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.592008+0000", 2024-04-25T20:09:30.584 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8819000000000001e-05 2024-04-25T20:09:30.584 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.584 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.584 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:30.584 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.594784+0000", 2024-04-25T20:09:30.584 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027750850000000001 2024-04-25T20:09:30.584 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.585 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.585 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:30.585 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.594822+0000", 2024-04-25T20:09:30.585 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8522000000000001e-05 2024-04-25T20:09:30.585 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.585 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:30.585 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.585 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.585 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.585 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:139 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'97, mlcod=17'87)", 2024-04-25T20:09:30.585 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:09:30.585 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 12.386166564, 2024-04-25T20:09:30.585 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048546290000000001, 2024-04-25T20:09:30.585 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:30.585 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:30.586 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:30.586 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.586 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:30.586 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:09:30.586 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.586 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.586 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.586 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:30.586 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:09:30.586 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.586 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.586 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.586 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:30.586 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125896+0000", 2024-04-25T20:09:30.587 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:30.587 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.587 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.587 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:30.587 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125908+0000", 2024-04-25T20:09:30.587 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.137e-05 2024-04-25T20:09:30.587 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.587 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.587 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:30.587 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125915+0000", 2024-04-25T20:09:30.587 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3939999999999998e-06 2024-04-25T20:09:30.587 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.587 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.587 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:30.588 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125922+0000", 2024-04-25T20:09:30.588 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1930000000000004e-06 2024-04-25T20:09:30.588 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.588 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.588 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:30.588 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125959+0000", 2024-04-25T20:09:30.588 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6701e-05 2024-04-25T20:09:30.588 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.588 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.588 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:30.588 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125972+0000", 2024-04-25T20:09:30.588 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2578000000000001e-05 2024-04-25T20:09:30.588 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.588 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.588 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:30.589 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130719+0000", 2024-04-25T20:09:30.589 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047475800000000004 2024-04-25T20:09:30.589 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.589 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.589 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:30.589 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130751+0000", 2024-04-25T20:09:30.589 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2283000000000002e-05 2024-04-25T20:09:30.589 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.589 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:30.589 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.589 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.589 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.589 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:140 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'98, mlcod=17'87)", 2024-04-25T20:09:30.589 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:09:30.590 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 12.385741212999999, 2024-04-25T20:09:30.590 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048310990000000002, 2024-04-25T20:09:30.590 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:30.590 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:30.590 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:30.590 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.590 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:30.590 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:09:30.590 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.590 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.590 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.590 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:30.590 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:09:30.590 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.590 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.591 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.591 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:30.591 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:09:30.591 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:30.591 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.591 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.591 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:30.591 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126427+0000", 2024-04-25T20:09:30.591 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000105572 2024-04-25T20:09:30.591 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.591 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.591 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:30.591 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126435+0000", 2024-04-25T20:09:30.591 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1920000000000005e-06 2024-04-25T20:09:30.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.592 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.592 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:30.592 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126439+0000", 2024-04-25T20:09:30.592 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6950000000000002e-06 2024-04-25T20:09:30.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.592 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.592 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:30.592 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126471+0000", 2024-04-25T20:09:30.592 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1699000000000003e-05 2024-04-25T20:09:30.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.592 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.592 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:30.592 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126484+0000", 2024-04-25T20:09:30.592 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.343e-05 2024-04-25T20:09:30.593 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.593 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.593 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:30.593 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131131+0000", 2024-04-25T20:09:30.593 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046471480000000003 2024-04-25T20:09:30.593 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.593 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.593 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:30.593 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131153+0000", 2024-04-25T20:09:30.593 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1753e-05 2024-04-25T20:09:30.593 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.593 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:30.593 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.593 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.594 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.594 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:141 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'99, mlcod=17'87)", 2024-04-25T20:09:30.594 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:09:30.594 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 12.385086269, 2024-04-25T20:09:30.594 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.004496847, 2024-04-25T20:09:30.594 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:30.594 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:30.594 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:30.594 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.595 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:30.595 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:09:30.595 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.595 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.595 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.595 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:30.595 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:09:30.595 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:30.595 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.595 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.595 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:30.595 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:09:30.595 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:30.595 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.596 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.596 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:30.596 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126980+0000", 2024-04-25T20:09:30.596 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.619e-06 2024-04-25T20:09:30.596 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.596 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.596 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:30.596 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126987+0000", 2024-04-25T20:09:30.596 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7220000000000002e-06 2024-04-25T20:09:30.596 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.596 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.596 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:30.596 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126992+0000", 2024-04-25T20:09:30.596 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8679999999999998e-06 2024-04-25T20:09:30.596 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.597 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.597 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:30.597 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130772+0000", 2024-04-25T20:09:30.597 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037803120000000001 2024-04-25T20:09:30.597 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.597 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.597 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:30.597 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130786+0000", 2024-04-25T20:09:30.597 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3592e-05 2024-04-25T20:09:30.597 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.597 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.597 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:30.597 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131451+0000", 2024-04-25T20:09:30.597 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000665206 2024-04-25T20:09:30.598 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:30.598 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:30.598 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:30.598 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131474+0000", 2024-04-25T20:09:30.598 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2957999999999999e-05 2024-04-25T20:09:30.598 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.598 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:30.598 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.598 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:30.598 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:30.598 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:30.599 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T20:09:30.613 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:30.613+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:30.714 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:30.714+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:30.746 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:30.746 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:30.746 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:09:30.746 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:30.756 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T20:09:30.814 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:30.813+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:30.861 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:30.861 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:30.861 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:09:30.861 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:09:30.861 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:30.869 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T20:09:30.915 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:30.915+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:30.963 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T20:09:31.016 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:31.014+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:31.016 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:31.016 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:09:31.016 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:09:31.016 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:09:31.016 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:31.025 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T20:09:31.091 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:09:31.091 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:09:31.108 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:09:31.109 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:09:31.117 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:31.117+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:31.125 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:09:31.125 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:09:31.141 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:09:31.142 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:09:31.159 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:09:31.159 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:09:31.174 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:31.174 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:31.177 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:09:31.177 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:09:31.184 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T20:09:31.195 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:09:31.195 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:09:31.214 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:09:31.215 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:09:31.218 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:31.216+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:31.301 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:31.301 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:31.309 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T20:09:31.353 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:31.352+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:31.415 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:31.415 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:31.419 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:31.419+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:31.424 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T20:09:31.520 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:31.520+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:31.537 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:31.537 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:31.547 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T20:09:31.621 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:31.620+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:31.689 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:31.689 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:31.697 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T20:09:31.722 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:31.722+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:31.798 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:31.798 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:31.806 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T20:09:31.823 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:31.823+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:31.907 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:31.907 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:31.916 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T20:09:31.925 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:31.925+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:32.016 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:32.016 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:32.025 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T20:09:32.025 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:32.025+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:32.082 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-25T20:09:32.126 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:32.126+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:32.166 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:32.167 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:32.175 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T20:09:32.227 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:32.227+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:32.316 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:32.317 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:32.326 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T20:09:32.328 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:32.328+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:32.429 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:32.429 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:32.429 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:32.428+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:32.437 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T20:09:32.529 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:32.529+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:32.537 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:32.537 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:32.546 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T20:09:32.631 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:32.630+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:32.647 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:32.647 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:32.656 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T20:09:32.732 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:32.732+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:32.759 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:32.759 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:32.768 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T20:09:32.833 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:32.833+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:32.870 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:32.870 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:32.879 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T20:09:32.933 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:32.933+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:32.992 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:32.992 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:33.001 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T20:09:33.042 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:33.042+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is unset 2024-04-25T20:09:33.087 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-25T20:09:33.087 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T20:09:33.087 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-04-25T20:09:33.087 INFO:tasks.thrashosds.thrasher:Setting osd 5 primary_affinity to 0.000000 2024-04-25T20:09:33.087 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 0 2024-04-25T20:09:33.136 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:33.136+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:33.152 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:33.152 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:33.161 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T20:09:33.224 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T20:09:33.269 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:33.269+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:33.273 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:33.273 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:33.283 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T20:09:33.352 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:09:33.352 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:09:33.353 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:33.352+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:33.369 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:09:33.369 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:09:33.387 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:09:33.387 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:09:33.404 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:09:33.404 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:09:33.421 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:09:33.422 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:09:33.433 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:33.433 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:33.433 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:09:33.433 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:33.438 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:33.438+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:33.439 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:09:33.439 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:09:33.442 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T20:09:33.456 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:09:33.457 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:09:33.473 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:09:33.474 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:09:33.540 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:33.540+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:33.562 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:33.563 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:33.563 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:09:33.563 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:09:33.563 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:33.571 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T20:09:33.641 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:33.640+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:33.679 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:33.679 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:09:33.679 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:09:33.679 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-04-25T20:09:33.679 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.679 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:30 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'16, mlcod=16'14)", 2024-04-25T20:09:33.679 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:09:33.679 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 16.510724192000001, 2024-04-25T20:09:33.679 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0054087670000000001, 2024-04-25T20:09:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:09:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:09:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.680 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167142+0000", 2024-04-25T20:09:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999976 2024-04-25T20:09:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167162+0000", 2024-04-25T20:09:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0109e-05 2024-04-25T20:09:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167188+0000", 2024-04-25T20:09:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6146e-05 2024-04-25T20:09:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.681 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.682 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:33.682 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167199+0000", 2024-04-25T20:09:33.682 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0767e-05 2024-04-25T20:09:33.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.682 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.682 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:33.682 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167256+0000", 2024-04-25T20:09:33.682 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7392000000000002e-05 2024-04-25T20:09:33.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.682 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.682 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:33.682 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167283+0000", 2024-04-25T20:09:33.682 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6577e-05 2024-04-25T20:09:33.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.172510+0000", 2024-04-25T20:09:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0052276759999999997 2024-04-25T20:09:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.172553+0000", 2024-04-25T20:09:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2444999999999998e-05 2024-04-25T20:09:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.683 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:31 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-04-25T20:09:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:09:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 16.510108114000001, 2024-04-25T20:09:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0055055479999999999, 2024-04-25T20:09:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:09:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:09:33.684 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:09:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167766+0000", 2024-04-25T20:09:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.122e-06 2024-04-25T20:09:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167776+0000", 2024-04-25T20:09:33.685 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0515e-05 2024-04-25T20:09:33.686 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.686 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.686 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:33.686 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167782+0000", 2024-04-25T20:09:33.686 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1149999999999999e-06 2024-04-25T20:09:33.686 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.686 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.686 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:33.686 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167837+0000", 2024-04-25T20:09:33.686 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4763999999999999e-05 2024-04-25T20:09:33.686 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.686 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.686 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:33.686 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167855+0000", 2024-04-25T20:09:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7589e-05 2024-04-25T20:09:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173221+0000", 2024-04-25T20:09:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.005365903 2024-04-25T20:09:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173266+0000", 2024-04-25T20:09:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5059999999999999e-05 2024-04-25T20:09:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:32 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-04-25T20:09:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:09:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 16.509732928999998, 2024-04-25T20:09:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0051558239999999998, 2024-04-25T20:09:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:09:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.688 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:09:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:09:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168173+0000", 2024-04-25T20:09:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7663999999999997e-05 2024-04-25T20:09:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.689 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168182+0000", 2024-04-25T20:09:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0650000000000005e-06 2024-04-25T20:09:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168188+0000", 2024-04-25T20:09:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0589999999999999e-06 2024-04-25T20:09:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168239+0000", 2024-04-25T20:09:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1467e-05 2024-04-25T20:09:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.690 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168260+0000", 2024-04-25T20:09:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0531e-05 2024-04-25T20:09:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173268+0000", 2024-04-25T20:09:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0050087980000000001 2024-04-25T20:09:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173291+0000", 2024-04-25T20:09:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.268e-05 2024-04-25T20:09:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.691 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:33 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'19, mlcod=16'14)", 2024-04-25T20:09:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:09:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 16.509222510000001, 2024-04-25T20:09:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046697049999999997, 2024-04-25T20:09:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:09:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:09:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168645+0000", 2024-04-25T20:09:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168651+0000", 2024-04-25T20:09:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1079999999999998e-06 2024-04-25T20:09:33.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.694 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.694 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:33.694 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168660+0000", 2024-04-25T20:09:33.694 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0010000000000007e-06 2024-04-25T20:09:33.694 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.694 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.694 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:33.694 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168666+0000", 2024-04-25T20:09:33.694 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7780000000000004e-06 2024-04-25T20:09:33.694 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.694 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.694 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:33.694 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168701+0000", 2024-04-25T20:09:33.694 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4490999999999999e-05 2024-04-25T20:09:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168729+0000", 2024-04-25T20:09:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8476999999999999e-05 2024-04-25T20:09:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173294+0000", 2024-04-25T20:09:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0045650730000000002 2024-04-25T20:09:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173315+0000", 2024-04-25T20:09:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1245000000000001e-05 2024-04-25T20:09:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:34 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'20, mlcod=16'14)", 2024-04-25T20:09:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:09:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 16.508723581000002, 2024-04-25T20:09:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041943309999999999, 2024-04-25T20:09:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.696 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:09:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:09:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169144+0000", 2024-04-25T20:09:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.697 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169150+0000", 2024-04-25T20:09:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9150000000000001e-06 2024-04-25T20:09:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169159+0000", 2024-04-25T20:09:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8899999999999996e-06 2024-04-25T20:09:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169169+0000", 2024-04-25T20:09:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0441e-05 2024-04-25T20:09:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.698 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169206+0000", 2024-04-25T20:09:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6816000000000001e-05 2024-04-25T20:09:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169234+0000", 2024-04-25T20:09:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8008999999999999e-05 2024-04-25T20:09:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173317+0000", 2024-04-25T20:09:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0040830160000000001 2024-04-25T20:09:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.699 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173339+0000", 2024-04-25T20:09:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1897000000000001e-05 2024-04-25T20:09:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:35 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'14)", 2024-04-25T20:09:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:09:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 16.508231261999999, 2024-04-25T20:09:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037236529999999999, 2024-04-25T20:09:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:33.700 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:09:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:09:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169636+0000", 2024-04-25T20:09:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.701 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.702 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:33.702 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169678+0000", 2024-04-25T20:09:33.702 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1752000000000003e-05 2024-04-25T20:09:33.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.702 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.702 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:33.702 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169688+0000", 2024-04-25T20:09:33.702 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0098000000000001e-05 2024-04-25T20:09:33.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.702 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.702 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:33.702 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169694+0000", 2024-04-25T20:09:33.702 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2349999999999996e-06 2024-04-25T20:09:33.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169733+0000", 2024-04-25T20:09:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8481e-05 2024-04-25T20:09:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169750+0000", 2024-04-25T20:09:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7509000000000001e-05 2024-04-25T20:09:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173340+0000", 2024-04-25T20:09:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0035898449999999999 2024-04-25T20:09:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.703 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173361+0000", 2024-04-25T20:09:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.039e-05 2024-04-25T20:09:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:36 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'22, mlcod=16'14)", 2024-04-25T20:09:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:09:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 16.507731201999999, 2024-04-25T20:09:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032472120000000002, 2024-04-25T20:09:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:33.704 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:09:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:09:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:09:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170152+0000", 2024-04-25T20:09:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5798e-05 2024-04-25T20:09:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170161+0000", 2024-04-25T20:09:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.064e-06 2024-04-25T20:09:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170167+0000", 2024-04-25T20:09:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8390000000000003e-06 2024-04-25T20:09:33.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:33.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170202+0000", 2024-04-25T20:09:33.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.5120999999999998e-05 2024-04-25T20:09:33.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:33.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170219+0000", 2024-04-25T20:09:33.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6929999999999999e-05 2024-04-25T20:09:33.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:33.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173362+0000", 2024-04-25T20:09:33.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0031427619999999999 2024-04-25T20:09:33.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.708 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:33.708 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173384+0000", 2024-04-25T20:09:33.708 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2237000000000001e-05 2024-04-25T20:09:33.708 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.708 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:33.708 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.708 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:40 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'26, mlcod=16'24)", 2024-04-25T20:09:33.708 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:09:33.708 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 16.500693290000001, 2024-04-25T20:09:33.708 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028727330000000001, 2024-04-25T20:09:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:09:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:09:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:33.709 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177173+0000", 2024-04-25T20:09:33.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999976 2024-04-25T20:09:33.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:33.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177186+0000", 2024-04-25T20:09:33.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3533000000000001e-05 2024-04-25T20:09:33.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:33.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177209+0000", 2024-04-25T20:09:33.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3280000000000001e-05 2024-04-25T20:09:33.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:33.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177220+0000", 2024-04-25T20:09:33.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0465000000000001e-05 2024-04-25T20:09:33.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.711 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:33.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177264+0000", 2024-04-25T20:09:33.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4413999999999999e-05 2024-04-25T20:09:33.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.711 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:33.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177289+0000", 2024-04-25T20:09:33.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4783999999999999e-05 2024-04-25T20:09:33.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.179990+0000", 2024-04-25T20:09:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027012939999999999 2024-04-25T20:09:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.180048+0000", 2024-04-25T20:09:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7299000000000001e-05 2024-04-25T20:09:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.712 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:67 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'41, mlcod=16'39)", 2024-04-25T20:09:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:09:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 16.465931798, 2024-04-25T20:09:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064192549999999996, 2024-04-25T20:09:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:09:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:09:33.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211935+0000", 2024-04-25T20:09:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999981 2024-04-25T20:09:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211948+0000", 2024-04-25T20:09:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3531e-05 2024-04-25T20:09:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:33.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211967+0000", 2024-04-25T20:09:33.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8326999999999999e-05 2024-04-25T20:09:33.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:33.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211975+0000", 2024-04-25T20:09:33.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8589999999999994e-06 2024-04-25T20:09:33.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:33.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212020+0000", 2024-04-25T20:09:33.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4626999999999998e-05 2024-04-25T20:09:33.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:33.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212045+0000", 2024-04-25T20:09:33.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4508999999999999e-05 2024-04-25T20:09:33.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:33.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218316+0000", 2024-04-25T20:09:33.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0062711640000000001 2024-04-25T20:09:33.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:33.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218356+0000", 2024-04-25T20:09:33.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9993000000000001e-05 2024-04-25T20:09:33.716 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.716 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:68 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-04-25T20:09:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:09:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 16.465482783999999, 2024-04-25T20:09:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011231365, 2024-04-25T20:09:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:09:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.717 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:09:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:09:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212392+0000", 2024-04-25T20:09:33.718 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.79e-06 2024-04-25T20:09:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212406+0000", 2024-04-25T20:09:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3577000000000001e-05 2024-04-25T20:09:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212414+0000", 2024-04-25T20:09:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6510000000000001e-06 2024-04-25T20:09:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217114+0000", 2024-04-25T20:09:33.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047005379999999998 2024-04-25T20:09:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217141+0000", 2024-04-25T20:09:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6984000000000001e-05 2024-04-25T20:09:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223563+0000", 2024-04-25T20:09:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064216999999999998 2024-04-25T20:09:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223617+0000", 2024-04-25T20:09:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4011000000000002e-05 2024-04-25T20:09:33.721 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.721 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:33.721 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.721 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.721 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:69 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-04-25T20:09:33.721 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:09:33.721 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 16.464950594000001, 2024-04-25T20:09:33.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.010730094000000001, 2024-04-25T20:09:33.721 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:33.721 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:33.722 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:33.722 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.722 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:33.722 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:09:33.722 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.722 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.722 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.722 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:33.722 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:09:33.722 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.722 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.722 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.722 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212917+0000", 2024-04-25T20:09:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212957+0000", 2024-04-25T20:09:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0692000000000003e-05 2024-04-25T20:09:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212973+0000", 2024-04-25T20:09:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.575e-05 2024-04-25T20:09:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:33.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212980+0000", 2024-04-25T20:09:33.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1679999999999998e-06 2024-04-25T20:09:33.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:33.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217449+0000", 2024-04-25T20:09:33.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.004468367 2024-04-25T20:09:33.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:33.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217470+0000", 2024-04-25T20:09:33.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0749000000000001e-05 2024-04-25T20:09:33.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223621+0000", 2024-04-25T20:09:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0061512880000000004 2024-04-25T20:09:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223648+0000", 2024-04-25T20:09:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6961000000000002e-05 2024-04-25T20:09:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:70 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'44, mlcod=16'39)", 2024-04-25T20:09:33.725 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:09:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 16.464191866, 2024-04-25T20:09:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.013553358, 2024-04-25T20:09:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:09:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:09:33.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.727 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:33.727 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:09:33.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:33.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.727 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:33.727 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213682+0000", 2024-04-25T20:09:33.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3439999999999997e-06 2024-04-25T20:09:33.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.727 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:33.727 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213693+0000", 2024-04-25T20:09:33.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1049999999999999e-05 2024-04-25T20:09:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213699+0000", 2024-04-25T20:09:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9399999999999999e-06 2024-04-25T20:09:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218086+0000", 2024-04-25T20:09:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0043871240000000001 2024-04-25T20:09:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218106+0000", 2024-04-25T20:09:33.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.997e-05 2024-04-25T20:09:33.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:33.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.227186+0000", 2024-04-25T20:09:33.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.009079574 2024-04-25T20:09:33.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:33.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.227230+0000", 2024-04-25T20:09:33.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4147999999999998e-05 2024-04-25T20:09:33.729 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.729 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:33.729 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.730 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.730 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:106 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-04-25T20:09:33.730 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:09:33.730 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 16.093052766, 2024-04-25T20:09:33.730 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002936756, 2024-04-25T20:09:33.730 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:33.730 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:33.730 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:33.730 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.730 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:33.730 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:09:33.730 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.730 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:09:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584815+0000", 2024-04-25T20:09:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584860+0000", 2024-04-25T20:09:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5161999999999999e-05 2024-04-25T20:09:33.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584874+0000", 2024-04-25T20:09:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4464000000000001e-05 2024-04-25T20:09:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584881+0000", 2024-04-25T20:09:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.409e-06 2024-04-25T20:09:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584915+0000", 2024-04-25T20:09:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4007000000000002e-05 2024-04-25T20:09:33.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:33.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584933+0000", 2024-04-25T20:09:33.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7737999999999999e-05 2024-04-25T20:09:33.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:33.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587717+0000", 2024-04-25T20:09:33.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002784711 2024-04-25T20:09:33.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:33.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587752+0000", 2024-04-25T20:09:33.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4915e-05 2024-04-25T20:09:33.734 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.734 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:33.734 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.734 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:107 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'69, mlcod=16'64)", 2024-04-25T20:09:33.734 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:09:33.734 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 16.092977464000001, 2024-04-25T20:09:33.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028906270000000002, 2024-04-25T20:09:33.734 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:33.734 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:33.734 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:33.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:33.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:09:33.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:33.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:09:33.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:33.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584890+0000", 2024-04-25T20:09:33.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:33.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584895+0000", 2024-04-25T20:09:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2069999999999997e-06 2024-04-25T20:09:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584906+0000", 2024-04-25T20:09:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.117e-05 2024-04-25T20:09:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584910+0000", 2024-04-25T20:09:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4190000000000004e-06 2024-04-25T20:09:33.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585310+0000", 2024-04-25T20:09:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00040002700000000001 2024-04-25T20:09:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585329+0000", 2024-04-25T20:09:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8838000000000002e-05 2024-04-25T20:09:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587754+0000", 2024-04-25T20:09:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024252689999999999 2024-04-25T20:09:33.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587781+0000", 2024-04-25T20:09:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7206999999999999e-05 2024-04-25T20:09:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:108 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'70, mlcod=16'64)", 2024-04-25T20:09:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:09:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 16.092952313000001, 2024-04-25T20:09:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028924879999999999, 2024-04-25T20:09:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:33.738 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:33.739 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:33.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.739 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:33.739 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:09:33.739 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.739 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:33.739 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:09:33.739 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.739 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:33.739 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:09:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584919+0000", 2024-04-25T20:09:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.557e-06 2024-04-25T20:09:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584925+0000", 2024-04-25T20:09:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8619999999999999e-06 2024-04-25T20:09:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.740 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584931+0000", 2024-04-25T20:09:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2539999999999998e-06 2024-04-25T20:09:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585550+0000", 2024-04-25T20:09:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00061821800000000002 2024-04-25T20:09:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585567+0000", 2024-04-25T20:09:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7499e-05 2024-04-25T20:09:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.741 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:33.742 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587783+0000", 2024-04-25T20:09:33.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0022156200000000002 2024-04-25T20:09:33.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:33.742 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587808+0000", 2024-04-25T20:09:33.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5729e-05 2024-04-25T20:09:33.742 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.742 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:33.742 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.742 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:112 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'74, mlcod=16'66)", 2024-04-25T20:09:33.742 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:09:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 16.090794690999999, 2024-04-25T20:09:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00318724, 2024-04-25T20:09:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:09:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:09:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587073+0000", 2024-04-25T20:09:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587136+0000", 2024-04-25T20:09:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2928999999999993e-05 2024-04-25T20:09:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587147+0000", 2024-04-25T20:09:33.744 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1182999999999999e-05 2024-04-25T20:09:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587153+0000", 2024-04-25T20:09:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9059999999999999e-06 2024-04-25T20:09:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587306+0000", 2024-04-25T20:09:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000152848 2024-04-25T20:09:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587330+0000", 2024-04-25T20:09:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4304e-05 2024-04-25T20:09:33.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:33.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.590230+0000", 2024-04-25T20:09:33.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029004629999999998 2024-04-25T20:09:33.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:33.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.590261+0000", 2024-04-25T20:09:33.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0323000000000001e-05 2024-04-25T20:09:33.746 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.746 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:33.746 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:118 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'80, mlcod=16'78)", 2024-04-25T20:09:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:09:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 16.086008111999998, 2024-04-25T20:09:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029618959999999999, 2024-04-25T20:09:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:09:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:33.747 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:09:33.748 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.748 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.748 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:33.748 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591859+0000", 2024-04-25T20:09:33.748 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:33.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.748 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.748 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:33.748 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591918+0000", 2024-04-25T20:09:33.748 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8842999999999999e-05 2024-04-25T20:09:33.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.748 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.748 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591933+0000", 2024-04-25T20:09:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4600000000000001e-05 2024-04-25T20:09:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591939+0000", 2024-04-25T20:09:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4559999999999997e-06 2024-04-25T20:09:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591990+0000", 2024-04-25T20:09:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0318999999999997e-05 2024-04-25T20:09:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.749 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:33.750 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.592008+0000", 2024-04-25T20:09:33.750 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8819000000000001e-05 2024-04-25T20:09:33.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.750 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.750 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:33.750 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.594784+0000", 2024-04-25T20:09:33.750 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027750850000000001 2024-04-25T20:09:33.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.750 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.750 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:33.750 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.594822+0000", 2024-04-25T20:09:33.750 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8522000000000001e-05 2024-04-25T20:09:33.750 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.750 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:33.751 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.751 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:139 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'97, mlcod=17'87)", 2024-04-25T20:09:33.751 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:09:33.751 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 15.551971506999999, 2024-04-25T20:09:33.751 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048546290000000001, 2024-04-25T20:09:33.751 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:33.751 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:33.751 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:33.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.751 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:33.751 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:09:33.751 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:09:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125896+0000", 2024-04-25T20:09:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125908+0000", 2024-04-25T20:09:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.137e-05 2024-04-25T20:09:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.752 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.753 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:33.753 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125915+0000", 2024-04-25T20:09:33.753 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3939999999999998e-06 2024-04-25T20:09:33.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.753 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.753 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:33.753 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125922+0000", 2024-04-25T20:09:33.753 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1930000000000004e-06 2024-04-25T20:09:33.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.753 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.753 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:33.753 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125959+0000", 2024-04-25T20:09:33.753 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6701e-05 2024-04-25T20:09:33.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.753 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125972+0000", 2024-04-25T20:09:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2578000000000001e-05 2024-04-25T20:09:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130719+0000", 2024-04-25T20:09:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047475800000000004 2024-04-25T20:09:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130751+0000", 2024-04-25T20:09:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2283000000000002e-05 2024-04-25T20:09:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:140 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'98, mlcod=17'87)", 2024-04-25T20:09:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:09:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 15.551546156000001, 2024-04-25T20:09:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048310990000000002, 2024-04-25T20:09:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:09:33.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:33.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:09:33.756 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:33.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:09:33.756 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:33.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:33.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126427+0000", 2024-04-25T20:09:33.757 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000105572 2024-04-25T20:09:33.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.757 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.757 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:33.757 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126435+0000", 2024-04-25T20:09:33.757 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1920000000000005e-06 2024-04-25T20:09:33.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.757 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.757 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:33.757 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126439+0000", 2024-04-25T20:09:33.757 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6950000000000002e-06 2024-04-25T20:09:33.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.757 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.757 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:33.758 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126471+0000", 2024-04-25T20:09:33.758 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1699000000000003e-05 2024-04-25T20:09:33.758 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.758 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.758 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:33.758 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126484+0000", 2024-04-25T20:09:33.758 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.343e-05 2024-04-25T20:09:33.758 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.758 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.758 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:33.758 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131131+0000", 2024-04-25T20:09:33.758 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046471480000000003 2024-04-25T20:09:33.758 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.758 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.759 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:33.759 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131153+0000", 2024-04-25T20:09:33.759 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1753e-05 2024-04-25T20:09:33.759 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.759 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:33.759 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.759 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.759 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.759 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:141 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'99, mlcod=17'87)", 2024-04-25T20:09:33.759 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:09:33.759 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 15.550891212, 2024-04-25T20:09:33.759 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.004496847, 2024-04-25T20:09:33.759 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:33.759 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:09:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:09:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:09:33.760 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126980+0000", 2024-04-25T20:09:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.619e-06 2024-04-25T20:09:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126987+0000", 2024-04-25T20:09:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7220000000000002e-06 2024-04-25T20:09:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126992+0000", 2024-04-25T20:09:33.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8679999999999998e-06 2024-04-25T20:09:33.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.762 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.762 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:33.762 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130772+0000", 2024-04-25T20:09:33.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037803120000000001 2024-04-25T20:09:33.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.762 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.762 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:33.762 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130786+0000", 2024-04-25T20:09:33.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3592e-05 2024-04-25T20:09:33.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.762 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.762 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:33.763 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131451+0000", 2024-04-25T20:09:33.763 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000665206 2024-04-25T20:09:33.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:33.763 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:33.763 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:33.763 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131474+0000", 2024-04-25T20:09:33.763 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2957999999999999e-05 2024-04-25T20:09:33.763 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.763 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:33.763 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.763 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:33.763 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:33.763 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:33.765 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T20:09:33.765 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:33.765+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:33.865 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:33.865+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:33.912 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:33.912 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:33.912 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:09:33.912 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:33.920 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T20:09:34.004 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:34.004+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:34.066 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:34.066 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:34.066 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:09:34.066 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:09:34.066 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:34.067 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:34.066+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:34.075 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T20:09:34.090 INFO:teuthology.orchestra.run.smithi039.stderr:set osd.5 primary-affinity to 0 (802) 2024-04-25T20:09:34.168 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:34.168+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:34.196 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:34.196 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:09:34.196 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:09:34.196 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:09:34.196 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:34.205 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T20:09:34.268 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:34.268+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:34.311 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:34.311 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:34.311 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:09:34.311 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:34.320 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T20:09:34.369 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:34.369+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:34.466 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:34.466 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:34.467 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:09:34.467 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:09:34.467 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:34.470 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:34.470+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:34.475 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T20:09:34.571 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:34.571+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:34.621 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:34.622 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:09:34.622 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:09:34.622 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:09:34.622 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:34.630 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T20:09:34.676 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:34.676+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:34.744 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:34.744 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:34.744 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:09:34.745 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:34.754 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T20:09:34.773 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:34.773+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:34.874 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:34.874+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:34.897 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:34.897 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:34.897 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:09:34.897 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:09:34.897 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:34.907 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T20:09:34.975 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:34.975+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:35.047 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:35.047 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:09:35.048 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:09:35.048 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:09:35.048 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:35.056 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T20:09:35.099 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-25T20:09:35.100 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:35.100+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:35.157 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:35.157 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:35.157 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:09:35.157 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:35.166 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T20:09:35.208 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:35.208+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:35.279 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:35.279+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:35.307 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:35.307 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:35.307 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:09:35.307 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:09:35.307 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:35.316 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T20:09:35.384 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:35.384+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:35.417 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:35.417 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:09:35.417 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:09:35.417 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:09:35.417 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:35.426 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T20:09:35.485 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T20:09:35.485 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:35.485+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:35.568 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:35.568 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:35.568 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:09:35.568 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:35.577 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T20:09:35.606 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:09:35.606 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:09:35.621 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:35.621+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:35.623 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:09:35.623 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:09:35.639 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:09:35.640 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:09:35.656 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:09:35.656 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:09:35.672 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:09:35.672 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:09:35.679 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:35.679 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:35.679 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:09:35.679 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:09:35.680 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:35.687 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:35.686+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:35.688 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T20:09:35.689 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:09:35.690 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:09:35.706 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:09:35.706 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:09:35.723 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:09:35.723 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:09:35.788 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:35.787+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:35.829 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:35.829 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:35.838 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T20:09:35.889 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:35.889+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:35.941 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:35.941 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:35.950 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T20:09:35.991 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:35.991+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:36.090 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:36.090+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:36.091 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:36.091 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:36.100 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T20:09:36.130 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is unset 2024-04-25T20:09:36.192 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:36.191+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:36.201 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:36.201 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:36.210 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T20:09:36.293 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:36.293+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:36.318 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:36.321 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:36.328 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T20:09:36.394 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:36.394+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:36.435 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:36.435 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:36.444 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T20:09:36.495 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:36.495+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:36.591 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:36.591 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:36.595 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:36.595+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:36.600 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T20:09:36.697 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:36.697+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:36.705 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:36.705 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:36.715 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T20:09:36.798 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:36.798+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:36.821 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:36.821 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:36.830 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T20:09:36.914 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:36.914+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:36.975 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:36.976 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:36.985 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T20:09:36.999 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:36.999+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:37.100 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:37.100+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:37.130 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:37.131 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:37.140 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T20:09:37.201 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:37.201+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:37.245 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:37.245 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:37.254 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T20:09:37.302 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:37.302+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:37.359 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:37.359 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:37.368 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T20:09:37.403 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:37.403+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:37.474 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:37.475 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:37.483 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T20:09:37.504 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:37.504+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:37.589 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:37.589 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:37.599 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T20:09:37.605 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:37.605+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:37.705 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:37.705 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:37.706 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:37.705+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:37.714 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T20:09:37.734 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T20:09:37.806 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:37.806+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:37.827 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:37.827 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:37.836 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T20:09:37.877 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:09:37.877 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:09:37.894 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:09:37.895 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:09:37.907 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:37.907+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:37.911 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:09:37.911 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:09:37.927 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:09:37.928 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:09:37.939 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:37.939 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:37.946 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:09:37.946 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:09:37.948 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T20:09:37.963 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:09:37.963 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:09:37.982 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:09:37.982 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:09:37.997 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:09:37.997 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:09:38.008 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:38.008+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:38.051 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:38.051 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:09:38.051 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:09:38.051 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:09:38.051 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:38.060 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T20:09:38.109 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:38.109+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:38.146 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-25T20:09:38.161 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:38.161 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:38.161 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:09:38.161 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:38.170 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T20:09:38.210 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:38.210+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:38.272 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:38.272 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:38.272 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:09:38.272 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:09:38.272 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:38.282 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T20:09:38.311 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:38.311+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:38.382 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:38.383 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:09:38.383 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:09:38.383 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:09:38.383 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:38.391 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T20:09:38.412 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:38.412+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:38.493 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:38.493 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:38.493 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:09:38.493 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:38.502 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T20:09:38.513 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:38.513+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:38.603 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:38.603 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:38.603 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:09:38.603 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:09:38.603 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:38.611 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T20:09:38.614 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:38.614+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:38.713 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:38.713 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:09:38.713 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:09:38.714 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:09:38.714 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:38.715 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:38.715+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:38.722 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T20:09:38.816 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:38.816+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:38.863 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:38.863 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:38.863 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:09:38.864 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:38.872 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T20:09:38.957 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:38.957+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:39.014 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:39.015 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:39.015 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:09:39.015 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:09:39.015 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:39.018 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:39.018+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:39.023 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T20:09:39.107 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-25T20:09:39.107 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T20:09:39.107 INFO:tasks.thrashosds.thrasher:Removing osd 3, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-25T20:09:39.107 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 3 2024-04-25T20:09:39.119 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:39.119+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:39.134 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is set 2024-04-25T20:09:39.169 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:39.169 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:09:39.169 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:09:39.170 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [ 2024-04-25T20:09:39.170 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.170 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:09:39.170 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:09:39.170 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 22.385528363999999, 2024-04-25T20:09:39.170 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.34154860100000001, 2024-04-25T20:09:39.170 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:39.170 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:39.170 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:39.170 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:39.170 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:39.170 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 1 2024-04-25T20:09:39.170 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.171 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:39.171 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.171 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:39.171 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:09:39.171 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.171 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.171 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.171 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:39.171 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:09:39.171 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.171 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.171 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.171 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:39.171 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782261+0000", 2024-04-25T20:09:39.172 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999986 2024-04-25T20:09:39.172 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.172 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.172 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:39.172 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782268+0000", 2024-04-25T20:09:39.172 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.8140000000000001e-06 2024-04-25T20:09:39.172 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.172 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.172 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:39.172 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782279+0000", 2024-04-25T20:09:39.172 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0665999999999999e-05 2024-04-25T20:09:39.172 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.172 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.172 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:39.173 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782290+0000", 2024-04-25T20:09:39.173 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.1343e-05 2024-04-25T20:09:39.173 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.173 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.173 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:39.173 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782340+0000", 2024-04-25T20:09:39.173 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.9311999999999998e-05 2024-04-25T20:09:39.173 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.173 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.173 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for peered", 2024-04-25T20:09:39.173 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782348+0000", 2024-04-25T20:09:39.173 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.0690000000000002e-06 2024-04-25T20:09:39.173 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.173 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.174 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:39.174 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120520+0000", 2024-04-25T20:09:39.174 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.33817188100000001 2024-04-25T20:09:39.174 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.174 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.174 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:39.174 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120656+0000", 2024-04-25T20:09:39.174 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000135897 2024-04-25T20:09:39.174 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.174 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.174 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:39.174 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120802+0000", 2024-04-25T20:09:39.174 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00014691199999999999 2024-04-25T20:09:39.174 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.175 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.175 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:39.175 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.122368+0000", 2024-04-25T20:09:39.175 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0015651040000000001 2024-04-25T20:09:39.175 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.175 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.175 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:39.175 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123729+0000", 2024-04-25T20:09:39.175 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0013610499999999999 2024-04-25T20:09:39.175 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.175 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.175 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:39.175 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123769+0000", 2024-04-25T20:09:39.175 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0719000000000001e-05 2024-04-25T20:09:39.175 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.176 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.176 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:39.176 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123811+0000", 2024-04-25T20:09:39.176 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1267999999999999e-05 2024-04-25T20:09:39.176 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.176 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:39.176 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.176 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.176 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.176 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:67 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:09:39.176 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:09:39.176 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 21.956109476999998, 2024-04-25T20:09:39.176 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.006947651, 2024-04-25T20:09:39.176 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:39.177 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:39.177 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:39.177 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:39.177 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:39.177 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 67 2024-04-25T20:09:39.177 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.177 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:39.177 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.177 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:39.177 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:09:39.177 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.177 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.177 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.178 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:39.178 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:09:39.178 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.178 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.178 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.178 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:39.178 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211680+0000", 2024-04-25T20:09:39.178 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:39.178 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.178 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.178 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:39.178 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211684+0000", 2024-04-25T20:09:39.178 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4249999999999998e-06 2024-04-25T20:09:39.178 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.179 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.179 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:39.179 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211690+0000", 2024-04-25T20:09:39.179 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.7360000000000004e-06 2024-04-25T20:09:39.179 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.179 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.179 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:39.179 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211694+0000", 2024-04-25T20:09:39.179 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4860000000000002e-06 2024-04-25T20:09:39.179 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.179 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.179 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:39.179 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211727+0000", 2024-04-25T20:09:39.179 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3445999999999998e-05 2024-04-25T20:09:39.179 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.180 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.180 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:39.180 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211778+0000", 2024-04-25T20:09:39.180 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1050000000000001e-05 2024-04-25T20:09:39.180 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.180 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.180 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:39.180 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211851+0000", 2024-04-25T20:09:39.180 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2914e-05 2024-04-25T20:09:39.180 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.180 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.180 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:39.180 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213824+0000", 2024-04-25T20:09:39.180 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0019726520000000001 2024-04-25T20:09:39.181 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.181 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.181 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:39.181 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218579+0000", 2024-04-25T20:09:39.181 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0047553159999999999 2024-04-25T20:09:39.181 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.181 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.181 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:39.181 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218607+0000", 2024-04-25T20:09:39.181 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8144999999999999e-05 2024-04-25T20:09:39.181 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.181 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.181 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:39.181 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218629+0000", 2024-04-25T20:09:39.182 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.1044e-05 2024-04-25T20:09:39.182 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.182 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:39.182 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.182 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.182 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.182 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:68 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:09:39.182 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:09:39.182 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 21.956091136000001, 2024-04-25T20:09:39.182 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012255596000000001, 2024-04-25T20:09:39.182 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:39.182 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:39.182 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:39.182 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:39.183 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:39.183 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 68 2024-04-25T20:09:39.183 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.183 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:39.183 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.183 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:39.183 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:09:39.183 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.183 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.183 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.183 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:39.183 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:09:39.183 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.183 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.184 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.184 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:39.184 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:09:39.184 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:39.184 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.184 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.184 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:39.184 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211701+0000", 2024-04-25T20:09:39.184 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2029999999999999e-06 2024-04-25T20:09:39.184 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.184 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.184 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:39.184 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211705+0000", 2024-04-25T20:09:39.184 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.2320000000000003e-06 2024-04-25T20:09:39.185 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.185 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.185 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:39.185 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211708+0000", 2024-04-25T20:09:39.185 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.6290000000000002e-06 2024-04-25T20:09:39.185 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.185 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.185 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:39.185 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212136+0000", 2024-04-25T20:09:39.185 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00042764099999999998 2024-04-25T20:09:39.185 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.185 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.185 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:39.185 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212212+0000", 2024-04-25T20:09:39.186 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.6315000000000004e-05 2024-04-25T20:09:39.186 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.186 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.186 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:39.186 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212306+0000", 2024-04-25T20:09:39.186 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.3556999999999999e-05 2024-04-25T20:09:39.186 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.186 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.186 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:39.186 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213829+0000", 2024-04-25T20:09:39.186 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001523256 2024-04-25T20:09:39.186 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.186 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.186 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:39.187 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223904+0000", 2024-04-25T20:09:39.187 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010075387 2024-04-25T20:09:39.187 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.187 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.187 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:39.187 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223934+0000", 2024-04-25T20:09:39.187 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9988000000000002e-05 2024-04-25T20:09:39.187 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.187 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.187 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:39.187 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223955+0000", 2024-04-25T20:09:39.187 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0630999999999999e-05 2024-04-25T20:09:39.187 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.187 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:39.187 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.188 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.188 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.188 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:69 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:09:39.188 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:09:39.188 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 21.956069401000001, 2024-04-25T20:09:39.188 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012316863000000001, 2024-04-25T20:09:39.188 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:39.188 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:39.188 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:39.188 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:39.188 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:39.188 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 69 2024-04-25T20:09:39.188 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.188 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:39.189 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.189 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:39.189 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:09:39.189 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.189 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.189 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.189 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:39.189 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:09:39.189 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.189 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.189 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.189 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:39.189 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:09:39.189 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:39.189 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.190 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.190 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:39.190 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211762+0000", 2024-04-25T20:09:39.190 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1703999999999999e-05 2024-04-25T20:09:39.190 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.190 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.190 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:39.190 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211768+0000", 2024-04-25T20:09:39.190 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.3430000000000003e-06 2024-04-25T20:09:39.190 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.190 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.190 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:39.190 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211771+0000", 2024-04-25T20:09:39.190 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.18e-06 2024-04-25T20:09:39.191 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.191 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.191 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:39.191 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212652+0000", 2024-04-25T20:09:39.191 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00088095800000000004 2024-04-25T20:09:39.191 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.191 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.191 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:39.191 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212703+0000", 2024-04-25T20:09:39.191 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.0655999999999997e-05 2024-04-25T20:09:39.191 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.191 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.191 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:39.191 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212781+0000", 2024-04-25T20:09:39.192 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.8852999999999999e-05 2024-04-25T20:09:39.192 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.192 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.192 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:39.192 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.214131+0000", 2024-04-25T20:09:39.192 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0013496039999999999 2024-04-25T20:09:39.192 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.192 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.192 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:39.192 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223986+0000", 2024-04-25T20:09:39.192 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.009854804 2024-04-25T20:09:39.192 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.192 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.192 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:39.193 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.224017+0000", 2024-04-25T20:09:39.193 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.1068999999999997e-05 2024-04-25T20:09:39.193 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.193 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.193 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:39.193 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.224038+0000", 2024-04-25T20:09:39.193 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0965e-05 2024-04-25T20:09:39.193 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.193 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:39.193 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.193 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.193 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.193 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:70 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:09:39.193 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:09:39.194 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 21.956014320000001, 2024-04-25T20:09:39.194 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.015776348999999999, 2024-04-25T20:09:39.194 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:39.194 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:39.194 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:39.194 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:39.194 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:39.194 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 70 2024-04-25T20:09:39.194 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.194 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:39.194 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.194 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:39.194 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:09:39.194 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.195 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.195 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.195 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:39.195 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:09:39.195 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.195 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.195 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.195 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:39.195 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:09:39.195 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:39.195 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.195 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.195 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:39.195 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211778+0000", 2024-04-25T20:09:39.196 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5330000000000001e-06 2024-04-25T20:09:39.196 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.196 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.196 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:39.196 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211782+0000", 2024-04-25T20:09:39.196 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4079999999999998e-06 2024-04-25T20:09:39.196 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.196 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.196 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:39.196 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211786+0000", 2024-04-25T20:09:39.196 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.6190000000000002e-06 2024-04-25T20:09:39.196 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.196 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.196 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:39.197 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213374+0000", 2024-04-25T20:09:39.197 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0015881859999999999 2024-04-25T20:09:39.197 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.197 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.197 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:39.197 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213478+0000", 2024-04-25T20:09:39.197 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000103464 2024-04-25T20:09:39.197 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.197 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.197 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:39.197 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213601+0000", 2024-04-25T20:09:39.197 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00012324300000000001 2024-04-25T20:09:39.197 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.199 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.199 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:39.199 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.214788+0000", 2024-04-25T20:09:39.199 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011868569999999999 2024-04-25T20:09:39.199 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.199 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.199 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:39.199 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227504+0000", 2024-04-25T20:09:39.199 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012715749 2024-04-25T20:09:39.199 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.199 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.200 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:39.200 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227532+0000", 2024-04-25T20:09:39.200 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.7937e-05 2024-04-25T20:09:39.200 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.200 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.200 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:39.200 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227552+0000", 2024-04-25T20:09:39.200 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0772999999999999e-05 2024-04-25T20:09:39.200 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.200 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:39.200 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.200 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.200 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.200 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:131 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:39.201 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:09:39.201 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 21.047004475000001, 2024-04-25T20:09:39.201 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0086158810000000006, 2024-04-25T20:09:39.201 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:39.201 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:39.201 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:39.201 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:39.201 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:39.201 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 131 2024-04-25T20:09:39.201 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.201 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:39.201 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.202 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:39.202 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:09:39.202 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.202 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.202 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.202 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:39.202 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:09:39.202 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.202 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.202 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.202 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:39.202 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120785+0000", 2024-04-25T20:09:39.202 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:39.203 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.203 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.203 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:39.203 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120880+0000", 2024-04-25T20:09:39.203 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.4925000000000003e-05 2024-04-25T20:09:39.203 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.203 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.203 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:39.203 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120891+0000", 2024-04-25T20:09:39.203 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0915e-05 2024-04-25T20:09:39.203 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.203 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.203 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:39.203 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120895+0000", 2024-04-25T20:09:39.204 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.8070000000000001e-06 2024-04-25T20:09:39.204 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.204 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.204 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:39.204 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121640+0000", 2024-04-25T20:09:39.206 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00074468500000000003 2024-04-25T20:09:39.206 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.206 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.206 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:39.206 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121684+0000", 2024-04-25T20:09:39.206 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4452000000000001e-05 2024-04-25T20:09:39.206 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.206 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.206 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:39.206 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121753+0000", 2024-04-25T20:09:39.206 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8508000000000002e-05 2024-04-25T20:09:39.206 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.206 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.207 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:39.207 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123618+0000", 2024-04-25T20:09:39.207 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0018655099999999999 2024-04-25T20:09:39.207 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.207 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.207 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:39.207 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129248+0000", 2024-04-25T20:09:39.207 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0056295540000000002 2024-04-25T20:09:39.207 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.207 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.207 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:39.207 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129368+0000", 2024-04-25T20:09:39.207 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000120244 2024-04-25T20:09:39.207 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.207 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.208 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:39.208 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129402+0000", 2024-04-25T20:09:39.208 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3812000000000001e-05 2024-04-25T20:09:39.208 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.208 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:39.208 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.208 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.208 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.208 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:132 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:39.208 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:09:39.208 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 21.046890264999998, 2024-04-25T20:09:39.208 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0085898169999999996, 2024-04-25T20:09:39.208 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:39.209 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:39.209 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:39.209 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:39.209 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:39.209 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 132 2024-04-25T20:09:39.209 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.209 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:39.209 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.209 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:39.209 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:09:39.209 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.209 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.209 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.209 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:39.210 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:09:39.210 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.210 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.210 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.210 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:39.210 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:09:39.210 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:39.210 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.210 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.210 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:39.210 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120902+0000", 2024-04-25T20:09:39.210 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.6129999999999998e-06 2024-04-25T20:09:39.210 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.211 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.211 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:39.211 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120906+0000", 2024-04-25T20:09:39.211 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6569999999999999e-06 2024-04-25T20:09:39.211 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.211 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.211 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:39.211 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120909+0000", 2024-04-25T20:09:39.211 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5550000000000001e-06 2024-04-25T20:09:39.211 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.211 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.211 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:39.211 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122098+0000", 2024-04-25T20:09:39.211 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011898049999999999 2024-04-25T20:09:39.211 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.212 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.212 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:39.212 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122172+0000", 2024-04-25T20:09:39.212 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3917000000000004e-05 2024-04-25T20:09:39.212 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.212 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.212 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:39.212 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122288+0000", 2024-04-25T20:09:39.212 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00011566 2024-04-25T20:09:39.212 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.212 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.212 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:39.212 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124519+0000", 2024-04-25T20:09:39.212 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0022308359999999999 2024-04-25T20:09:39.212 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.212 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.213 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:39.213 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129414+0000", 2024-04-25T20:09:39.213 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048953720000000003 2024-04-25T20:09:39.213 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.213 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.213 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:39.213 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129458+0000", 2024-04-25T20:09:39.213 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.3426000000000001e-05 2024-04-25T20:09:39.213 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.213 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.213 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:39.213 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129490+0000", 2024-04-25T20:09:39.213 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2329999999999997e-05 2024-04-25T20:09:39.213 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.214 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:39.214 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.214 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.214 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.214 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:133 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:39.214 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:09:39.214 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 21.046836396, 2024-04-25T20:09:39.214 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0086066270000000004, 2024-04-25T20:09:39.214 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:39.214 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:39.214 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:39.214 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:39.214 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:39.214 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 133 2024-04-25T20:09:39.215 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.215 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:39.215 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.215 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:39.215 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:09:39.215 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.215 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.215 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.215 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:39.215 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:09:39.215 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.215 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.215 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.215 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:39.216 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:09:39.216 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:39.216 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.216 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.216 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:39.216 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120956+0000", 2024-04-25T20:09:39.216 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5289999999999998e-06 2024-04-25T20:09:39.216 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.216 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.216 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:39.216 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120962+0000", 2024-04-25T20:09:39.216 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.9490000000000001e-06 2024-04-25T20:09:39.216 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.216 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.217 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:39.217 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120965+0000", 2024-04-25T20:09:39.217 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.1860000000000001e-06 2024-04-25T20:09:39.217 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.217 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.217 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:39.217 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122666+0000", 2024-04-25T20:09:39.217 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0017004500000000001 2024-04-25T20:09:39.217 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.217 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.217 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:39.217 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122741+0000", 2024-04-25T20:09:39.217 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.5297999999999996e-05 2024-04-25T20:09:39.217 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.217 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.217 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:39.218 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122858+0000", 2024-04-25T20:09:39.218 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00011723899999999999 2024-04-25T20:09:39.218 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.218 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.218 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:39.218 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124625+0000", 2024-04-25T20:09:39.218 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0017670100000000001 2024-04-25T20:09:39.218 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.218 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.218 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:39.218 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129497+0000", 2024-04-25T20:09:39.218 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048713580000000001 2024-04-25T20:09:39.218 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.218 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.219 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:39.219 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129532+0000", 2024-04-25T20:09:39.219 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5704999999999998e-05 2024-04-25T20:09:39.219 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.219 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.219 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:39.219 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129561+0000", 2024-04-25T20:09:39.219 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8283e-05 2024-04-25T20:09:39.219 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.219 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:39.219 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.219 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.219 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.219 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:134 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:39.220 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:09:39.220 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 21.046734197999999, 2024-04-25T20:09:39.220 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0085771179999999999, 2024-04-25T20:09:39.220 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:39.220 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:39.220 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:39.220 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:39.220 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:39.220 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 134 2024-04-25T20:09:39.220 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.220 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:39.220 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.220 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:39.220 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:09:39.221 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.221 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.221 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.221 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:39.221 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:09:39.221 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.221 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.221 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.221 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:39.221 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:09:39.221 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:39.221 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.221 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.221 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:39.222 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121157+0000", 2024-04-25T20:09:39.222 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00010104599999999999 2024-04-25T20:09:39.222 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.222 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.222 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:39.222 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121189+0000", 2024-04-25T20:09:39.222 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2589000000000003e-05 2024-04-25T20:09:39.222 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.222 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.222 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:39.222 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121195+0000", 2024-04-25T20:09:39.222 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.083e-06 2024-04-25T20:09:39.222 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.222 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.223 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:39.223 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123136+0000", 2024-04-25T20:09:39.223 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0019409780000000001 2024-04-25T20:09:39.223 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.223 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.223 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:39.223 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123181+0000", 2024-04-25T20:09:39.223 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4267999999999997e-05 2024-04-25T20:09:39.223 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.223 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.223 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:39.223 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123247+0000", 2024-04-25T20:09:39.223 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.6086000000000004e-05 2024-04-25T20:09:39.224 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.224 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.224 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:39.224 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125639+0000", 2024-04-25T20:09:39.224 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.002392 2024-04-25T20:09:39.224 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.224 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.224 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:39.224 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129567+0000", 2024-04-25T20:09:39.224 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0039282170000000003 2024-04-25T20:09:39.224 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.224 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.224 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:39.225 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129605+0000", 2024-04-25T20:09:39.225 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.7660999999999997e-05 2024-04-25T20:09:39.225 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.225 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.225 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:39.225 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129633+0000", 2024-04-25T20:09:39.225 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8649999999999998e-05 2024-04-25T20:09:39.225 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.225 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:39.225 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.225 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.225 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.225 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:135 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:39.225 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:09:39.226 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 21.046575191999999, 2024-04-25T20:09:39.226 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010898453000000001, 2024-04-25T20:09:39.226 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:39.226 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:39.226 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:39.226 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:39.226 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:39.226 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 135 2024-04-25T20:09:39.226 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.226 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:39.226 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.226 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:39.226 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:09:39.227 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.227 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.227 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.227 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:39.227 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:09:39.227 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.227 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.227 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.227 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:39.227 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121214+0000", 2024-04-25T20:09:39.227 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:39.227 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.227 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.227 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:39.228 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121218+0000", 2024-04-25T20:09:39.228 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6619999999999998e-06 2024-04-25T20:09:39.228 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.228 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.228 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:39.228 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121223+0000", 2024-04-25T20:09:39.228 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1070000000000004e-06 2024-04-25T20:09:39.228 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.228 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.228 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:39.228 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121227+0000", 2024-04-25T20:09:39.228 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2820000000000001e-06 2024-04-25T20:09:39.228 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.229 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.229 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:39.229 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123649+0000", 2024-04-25T20:09:39.229 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0024221970000000001 2024-04-25T20:09:39.229 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.229 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.229 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:39.229 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123723+0000", 2024-04-25T20:09:39.229 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.4386000000000002e-05 2024-04-25T20:09:39.229 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.229 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.229 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:39.229 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123844+0000", 2024-04-25T20:09:39.229 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000121249 2024-04-25T20:09:39.230 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.230 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.230 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:39.230 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125670+0000", 2024-04-25T20:09:39.230 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001825901 2024-04-25T20:09:39.230 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.230 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.230 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:39.230 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132043+0000", 2024-04-25T20:09:39.230 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0063724539999999996 2024-04-25T20:09:39.230 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.230 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.230 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:39.231 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132084+0000", 2024-04-25T20:09:39.231 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0963000000000001e-05 2024-04-25T20:09:39.231 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.231 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.231 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:39.231 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132114+0000", 2024-04-25T20:09:39.231 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9927e-05 2024-04-25T20:09:39.231 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.231 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:39.231 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.231 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.231 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.231 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:136 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:39.231 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:09:39.232 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 21.046559666, 2024-04-25T20:09:39.232 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010952513000000001, 2024-04-25T20:09:39.232 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:39.232 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:39.232 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:39.232 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:39.232 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:39.232 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 136 2024-04-25T20:09:39.232 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.232 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:39.232 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.232 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:39.232 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:09:39.232 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.233 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.233 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.233 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:39.233 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:09:39.233 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.233 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.233 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.233 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:39.233 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121230+0000", 2024-04-25T20:09:39.233 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:39.233 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.233 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.233 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:39.233 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121236+0000", 2024-04-25T20:09:39.233 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.7320000000000003e-06 2024-04-25T20:09:39.234 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.234 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.234 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:39.234 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121242+0000", 2024-04-25T20:09:39.234 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.7810000000000002e-06 2024-04-25T20:09:39.234 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.234 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.234 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:39.234 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121246+0000", 2024-04-25T20:09:39.234 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.639e-06 2024-04-25T20:09:39.234 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.234 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.234 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:39.235 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124150+0000", 2024-04-25T20:09:39.235 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.002903816 2024-04-25T20:09:39.235 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.235 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.235 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:39.235 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124195+0000", 2024-04-25T20:09:39.235 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.5042e-05 2024-04-25T20:09:39.235 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.235 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.235 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:39.235 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124266+0000", 2024-04-25T20:09:39.235 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0234999999999994e-05 2024-04-25T20:09:39.235 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.235 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.235 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:39.236 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125711+0000", 2024-04-25T20:09:39.236 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0014458800000000001 2024-04-25T20:09:39.236 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.236 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.236 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:39.236 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132120+0000", 2024-04-25T20:09:39.236 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0064084399999999996 2024-04-25T20:09:39.236 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.236 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.236 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:39.236 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132155+0000", 2024-04-25T20:09:39.236 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4938000000000003e-05 2024-04-25T20:09:39.236 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.236 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.236 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:39.237 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132183+0000", 2024-04-25T20:09:39.237 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8360999999999998e-05 2024-04-25T20:09:39.237 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.237 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:39.237 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.237 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.237 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.237 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:39.237 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:09:39.237 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 21.046509933999999, 2024-04-25T20:09:39.237 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.01097684, 2024-04-25T20:09:39.237 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:39.237 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:39.237 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:39.238 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:39.238 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:39.238 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 137 2024-04-25T20:09:39.238 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.238 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:39.238 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.238 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:39.238 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:09:39.238 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.238 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.238 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.238 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:39.238 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:09:39.239 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.239 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.239 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.239 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:39.239 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:09:39.239 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:39.239 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.239 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.239 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:39.239 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121328+0000", 2024-04-25T20:09:39.239 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.7908999999999998e-05 2024-04-25T20:09:39.239 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.239 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.239 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:39.240 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121337+0000", 2024-04-25T20:09:39.240 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.9369999999999993e-06 2024-04-25T20:09:39.240 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.240 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.240 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:39.240 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121343+0000", 2024-04-25T20:09:39.240 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.8089999999999998e-06 2024-04-25T20:09:39.240 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.240 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.240 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:39.240 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124656+0000", 2024-04-25T20:09:39.240 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0033133839999999999 2024-04-25T20:09:39.240 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.240 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.241 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:39.241 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124730+0000", 2024-04-25T20:09:39.241 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3565000000000005e-05 2024-04-25T20:09:39.241 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.241 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.241 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:39.241 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124846+0000", 2024-04-25T20:09:39.241 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000116416 2024-04-25T20:09:39.241 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.241 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.241 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:39.241 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126756+0000", 2024-04-25T20:09:39.241 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001909991 2024-04-25T20:09:39.241 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.242 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.242 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:39.242 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132190+0000", 2024-04-25T20:09:39.242 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0054343880000000001 2024-04-25T20:09:39.242 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.242 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.242 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:39.242 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132227+0000", 2024-04-25T20:09:39.242 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6875000000000002e-05 2024-04-25T20:09:39.242 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.242 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.242 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:39.242 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132257+0000", 2024-04-25T20:09:39.242 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.0046e-05 2024-04-25T20:09:39.243 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.243 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:39.243 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.243 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.243 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.243 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:39.243 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:09:39.243 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 21.046439975999998, 2024-04-25T20:09:39.243 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010978653999999999, 2024-04-25T20:09:39.243 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:39.243 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:39.243 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:39.243 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:39.244 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:39.244 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 138 2024-04-25T20:09:39.244 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.244 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:39.244 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.244 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:39.244 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:09:39.244 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.244 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.244 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.244 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:39.244 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:09:39.245 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.245 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.245 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.245 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:39.245 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:09:39.245 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:39.245 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.245 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.245 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:39.245 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121355+0000", 2024-04-25T20:09:39.245 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.9869999999999999e-06 2024-04-25T20:09:39.245 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.245 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.246 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:39.246 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121361+0000", 2024-04-25T20:09:39.246 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.4620000000000001e-06 2024-04-25T20:09:39.246 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.246 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.246 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:39.246 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121373+0000", 2024-04-25T20:09:39.246 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.2035000000000001e-05 2024-04-25T20:09:39.246 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.246 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.246 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:39.246 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125267+0000", 2024-04-25T20:09:39.247 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0038939230000000001 2024-04-25T20:09:39.247 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.247 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.247 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:39.247 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125309+0000", 2024-04-25T20:09:39.247 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1690999999999998e-05 2024-04-25T20:09:39.247 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.247 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.247 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:39.247 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125385+0000", 2024-04-25T20:09:39.247 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.6162999999999996e-05 2024-04-25T20:09:39.247 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.247 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.247 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:39.248 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126798+0000", 2024-04-25T20:09:39.248 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0014124680000000001 2024-04-25T20:09:39.248 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.248 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.248 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:39.248 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132264+0000", 2024-04-25T20:09:39.248 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0054661689999999999 2024-04-25T20:09:39.248 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.248 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.248 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:39.248 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132299+0000", 2024-04-25T20:09:39.248 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5272999999999999e-05 2024-04-25T20:09:39.248 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.248 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.249 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:39.249 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132329+0000", 2024-04-25T20:09:39.249 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.0080999999999999e-05 2024-04-25T20:09:39.249 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.249 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:39.249 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.249 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.249 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.249 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:39.249 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:09:39.249 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 21.046409400999998, 2024-04-25T20:09:39.249 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011015853000000001, 2024-04-25T20:09:39.249 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:39.249 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:39.250 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:39.250 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:39.250 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:39.250 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 139 2024-04-25T20:09:39.250 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.250 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:39.250 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.250 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:39.250 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:09:39.250 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.250 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.250 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.250 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:39.250 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:09:39.251 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.251 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.251 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.251 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:39.251 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121380+0000", 2024-04-25T20:09:39.251 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:39.251 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.251 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.251 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:39.251 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121385+0000", 2024-04-25T20:09:39.251 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.092e-06 2024-04-25T20:09:39.251 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.251 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.251 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:39.251 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121390+0000", 2024-04-25T20:09:39.252 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.5990000000000001e-06 2024-04-25T20:09:39.252 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.252 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.252 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:39.252 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121397+0000", 2024-04-25T20:09:39.252 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.6279999999999999e-06 2024-04-25T20:09:39.252 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.252 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.252 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:39.252 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125737+0000", 2024-04-25T20:09:39.252 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0043406010000000004 2024-04-25T20:09:39.252 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.252 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.252 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:39.253 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125781+0000", 2024-04-25T20:09:39.253 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.3958999999999997e-05 2024-04-25T20:09:39.253 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.253 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.253 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:39.253 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125849+0000", 2024-04-25T20:09:39.253 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8071999999999995e-05 2024-04-25T20:09:39.253 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.253 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.253 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:39.253 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.130908+0000", 2024-04-25T20:09:39.253 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0050586509999999999 2024-04-25T20:09:39.253 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.253 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.254 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:39.254 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132334+0000", 2024-04-25T20:09:39.254 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00142621 2024-04-25T20:09:39.254 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.254 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.254 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:39.254 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132369+0000", 2024-04-25T20:09:39.254 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4452000000000002e-05 2024-04-25T20:09:39.254 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.254 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.254 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:39.254 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132397+0000", 2024-04-25T20:09:39.254 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8065e-05 2024-04-25T20:09:39.255 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.255 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:39.255 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.255 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.255 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.255 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:39.255 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:09:39.255 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 21.046388357000001, 2024-04-25T20:09:39.255 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011069443, 2024-04-25T20:09:39.255 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:39.255 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:39.255 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:39.255 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:39.255 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:39.256 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 140 2024-04-25T20:09:39.256 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.256 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:39.256 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.256 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:39.256 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:09:39.256 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.256 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.256 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.256 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:39.256 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:09:39.256 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.256 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.256 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.257 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:39.257 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:09:39.257 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:39.257 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.257 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.257 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:39.257 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121456+0000", 2024-04-25T20:09:39.257 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.4469000000000003e-05 2024-04-25T20:09:39.257 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.257 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.257 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:39.257 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121462+0000", 2024-04-25T20:09:39.257 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.1890000000000004e-06 2024-04-25T20:09:39.258 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.258 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.258 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:39.258 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121466+0000", 2024-04-25T20:09:39.258 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3909999999999998e-06 2024-04-25T20:09:39.258 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.258 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.258 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:39.258 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126071+0000", 2024-04-25T20:09:39.258 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0046048979999999996 2024-04-25T20:09:39.258 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.258 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.258 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:39.259 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126141+0000", 2024-04-25T20:09:39.259 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0621e-05 2024-04-25T20:09:39.259 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.259 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.259 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:39.259 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126260+0000", 2024-04-25T20:09:39.259 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000118621 2024-04-25T20:09:39.259 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.259 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.259 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:39.259 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.131270+0000", 2024-04-25T20:09:39.259 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0050096949999999998 2024-04-25T20:09:39.259 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.260 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.260 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:39.260 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132403+0000", 2024-04-25T20:09:39.260 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011332779999999999 2024-04-25T20:09:39.260 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.260 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.260 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:39.260 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132443+0000", 2024-04-25T20:09:39.260 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0175000000000001e-05 2024-04-25T20:09:39.260 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.260 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.260 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:39.261 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132471+0000", 2024-04-25T20:09:39.261 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8399e-05 2024-04-25T20:09:39.261 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.261 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:39.261 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.261 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.261 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.261 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:39.261 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:09:39.261 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 21.046319572000002, 2024-04-25T20:09:39.261 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011071002, 2024-04-25T20:09:39.261 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:39.261 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:39.261 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:39.262 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:39.262 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:39.262 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 141 2024-04-25T20:09:39.262 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.262 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:39.262 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.262 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:39.262 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:09:39.262 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.262 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.262 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.262 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:39.262 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:09:39.262 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.263 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.263 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.263 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:39.263 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121470+0000", 2024-04-25T20:09:39.263 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:39.263 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.263 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.263 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:39.263 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121473+0000", 2024-04-25T20:09:39.263 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.01e-06 2024-04-25T20:09:39.263 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.263 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.263 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:39.263 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121478+0000", 2024-04-25T20:09:39.264 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.515e-06 2024-04-25T20:09:39.264 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.264 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.264 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:39.264 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121480+0000", 2024-04-25T20:09:39.264 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5749999999999999e-06 2024-04-25T20:09:39.264 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.264 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.264 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:39.264 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126813+0000", 2024-04-25T20:09:39.264 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0053320399999999997 2024-04-25T20:09:39.264 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.264 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.264 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:39.264 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126854+0000", 2024-04-25T20:09:39.265 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1582999999999998e-05 2024-04-25T20:09:39.265 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.265 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.265 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:39.265 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126923+0000", 2024-04-25T20:09:39.265 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8544e-05 2024-04-25T20:09:39.265 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.265 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.265 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:39.265 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.131573+0000", 2024-04-25T20:09:39.265 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0046504659999999998 2024-04-25T20:09:39.265 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.265 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.265 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:39.266 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132478+0000", 2024-04-25T20:09:39.266 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00090442400000000003 2024-04-25T20:09:39.266 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.266 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.266 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:39.266 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132513+0000", 2024-04-25T20:09:39.266 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5204999999999999e-05 2024-04-25T20:09:39.266 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.266 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.266 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:39.266 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132542+0000", 2024-04-25T20:09:39.266 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.904e-05 2024-04-25T20:09:39.266 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.266 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:39.267 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.267 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.267 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.267 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:156 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:39.267 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:09:39.267 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 20.548495277000001, 2024-04-25T20:09:39.267 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090072280000000008, 2024-04-25T20:09:39.267 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:39.267 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:39.267 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:39.267 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:39.267 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:39.267 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 156 2024-04-25T20:09:39.267 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.268 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:39.268 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.268 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:39.268 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:09:39.268 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.268 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.268 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.268 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:39.268 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:09:39.268 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.268 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.268 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.268 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:39.269 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:09:39.269 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:39.269 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.269 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.269 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:39.269 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619297+0000", 2024-04-25T20:09:39.269 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2979999999999999e-06 2024-04-25T20:09:39.269 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.269 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.269 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:39.269 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619300+0000", 2024-04-25T20:09:39.269 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.371e-06 2024-04-25T20:09:39.269 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.270 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.270 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:39.270 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619304+0000", 2024-04-25T20:09:39.270 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.304e-06 2024-04-25T20:09:39.270 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.270 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.270 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:39.270 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.622910+0000", 2024-04-25T20:09:39.270 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0036065049999999999 2024-04-25T20:09:39.270 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.270 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.270 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:39.270 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.622990+0000", 2024-04-25T20:09:39.271 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.9722999999999995e-05 2024-04-25T20:09:39.271 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.271 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.271 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:39.271 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623128+0000", 2024-04-25T20:09:39.271 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00013818399999999999 2024-04-25T20:09:39.271 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.271 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.271 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:39.271 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625155+0000", 2024-04-25T20:09:39.271 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0020267979999999998 2024-04-25T20:09:39.271 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.271 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.271 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:39.271 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628225+0000", 2024-04-25T20:09:39.272 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0030705680000000001 2024-04-25T20:09:39.272 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.272 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.272 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:39.272 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628268+0000", 2024-04-25T20:09:39.272 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.2404000000000003e-05 2024-04-25T20:09:39.272 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.272 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.272 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:39.272 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628302+0000", 2024-04-25T20:09:39.272 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4439e-05 2024-04-25T20:09:39.272 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.272 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:39.272 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.273 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.273 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.273 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:157 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:39.273 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:09:39.273 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 20.548483178000001, 2024-04-25T20:09:39.273 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090534889999999996, 2024-04-25T20:09:39.273 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:39.273 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:39.273 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:39.273 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:39.273 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:39.273 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 157 2024-04-25T20:09:39.273 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.273 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:39.273 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.274 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:39.274 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:09:39.274 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.274 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.274 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.274 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:39.274 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:09:39.274 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.274 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.274 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.274 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:39.274 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:09:39.274 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:39.275 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.275 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.275 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:39.275 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619309+0000", 2024-04-25T20:09:39.275 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.841e-06 2024-04-25T20:09:39.275 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.275 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.275 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:39.275 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619312+0000", 2024-04-25T20:09:39.275 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.084e-06 2024-04-25T20:09:39.275 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.275 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.275 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:39.275 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619314+0000", 2024-04-25T20:09:39.275 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.3860000000000001e-06 2024-04-25T20:09:39.275 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.276 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.276 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:39.276 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623718+0000", 2024-04-25T20:09:39.276 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0044038050000000002 2024-04-25T20:09:39.276 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.276 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.276 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:39.276 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623791+0000", 2024-04-25T20:09:39.276 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3181000000000003e-05 2024-04-25T20:09:39.276 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.276 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.276 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:39.276 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623922+0000", 2024-04-25T20:09:39.277 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00013117000000000001 2024-04-25T20:09:39.277 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.277 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.277 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:39.277 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625187+0000", 2024-04-25T20:09:39.277 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001264961 2024-04-25T20:09:39.277 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.277 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.277 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:39.277 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628308+0000", 2024-04-25T20:09:39.277 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.003120739 2024-04-25T20:09:39.277 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.277 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.277 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:39.278 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628331+0000", 2024-04-25T20:09:39.278 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2802999999999998e-05 2024-04-25T20:09:39.278 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.278 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.278 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:39.278 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628361+0000", 2024-04-25T20:09:39.278 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9760999999999998e-05 2024-04-25T20:09:39.278 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.278 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:39.278 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.278 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.278 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.278 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:158 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:39.279 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:09:39.279 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 20.548472731, 2024-04-25T20:09:39.279 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0091076409999999997, 2024-04-25T20:09:39.279 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:39.279 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:39.279 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:39.279 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:39.279 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:39.279 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 158 2024-04-25T20:09:39.279 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.279 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:39.279 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.279 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:39.279 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:09:39.280 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.280 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.280 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.280 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:39.280 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:09:39.280 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.280 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.280 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.280 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:39.280 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619317+0000", 2024-04-25T20:09:39.280 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967296 2024-04-25T20:09:39.280 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.280 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.280 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:39.280 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619364+0000", 2024-04-25T20:09:39.281 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.6572000000000001e-05 2024-04-25T20:09:39.281 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.281 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.281 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:39.281 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619371+0000", 2024-04-25T20:09:39.281 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3860000000000001e-06 2024-04-25T20:09:39.281 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.281 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.281 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:39.281 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619382+0000", 2024-04-25T20:09:39.281 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0173999999999999e-05 2024-04-25T20:09:39.281 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.281 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.281 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:39.282 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624216+0000", 2024-04-25T20:09:39.282 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048340350000000004 2024-04-25T20:09:39.282 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.282 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.282 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:39.282 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624260+0000", 2024-04-25T20:09:39.282 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4558999999999998e-05 2024-04-25T20:09:39.282 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.282 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.282 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:39.282 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624333+0000", 2024-04-25T20:09:39.282 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2744999999999996e-05 2024-04-25T20:09:39.282 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.283 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.283 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:39.283 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625465+0000", 2024-04-25T20:09:39.283 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001132454 2024-04-25T20:09:39.283 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.283 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.283 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:39.283 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628365+0000", 2024-04-25T20:09:39.283 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0028994950000000002 2024-04-25T20:09:39.283 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.283 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.283 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:39.283 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628386+0000", 2024-04-25T20:09:39.283 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.1367000000000001e-05 2024-04-25T20:09:39.284 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.284 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.284 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:39.284 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628425+0000", 2024-04-25T20:09:39.284 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.9087e-05 2024-04-25T20:09:39.284 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.284 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:39.284 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.284 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.284 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.284 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:159 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:39.284 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:09:39.284 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 20.548404078000001, 2024-04-25T20:09:39.284 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090948999999999995, 2024-04-25T20:09:39.284 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:39.285 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:39.285 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:39.285 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:39.285 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:39.285 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 159 2024-04-25T20:09:39.285 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.285 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:39.285 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.285 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:39.285 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:09:39.285 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.285 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.286 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.286 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:39.286 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:09:39.286 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:39.286 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.286 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.286 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:39.286 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:09:39.286 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:39.286 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.286 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.286 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:39.286 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619389+0000", 2024-04-25T20:09:39.286 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.734e-06 2024-04-25T20:09:39.287 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.287 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.287 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:39.287 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619392+0000", 2024-04-25T20:09:39.287 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6219999999999998e-06 2024-04-25T20:09:39.287 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.287 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.287 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:39.287 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619403+0000", 2024-04-25T20:09:39.287 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0987e-05 2024-04-25T20:09:39.287 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.287 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.287 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:39.288 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624667+0000", 2024-04-25T20:09:39.288 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0052633970000000004 2024-04-25T20:09:39.288 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.288 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.288 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:39.288 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624741+0000", 2024-04-25T20:09:39.288 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.4583000000000001e-05 2024-04-25T20:09:39.288 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.288 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.288 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:39.288 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624857+0000", 2024-04-25T20:09:39.288 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000116118 2024-04-25T20:09:39.288 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.288 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.289 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:39.289 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.626013+0000", 2024-04-25T20:09:39.289 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001155473 2024-04-25T20:09:39.289 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.289 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.289 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:39.289 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628432+0000", 2024-04-25T20:09:39.289 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0024189379999999998 2024-04-25T20:09:39.289 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.289 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.289 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:39.289 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628454+0000", 2024-04-25T20:09:39.289 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2059999999999999e-05 2024-04-25T20:09:39.289 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:39.290 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:39.290 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:39.290 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628481+0000", 2024-04-25T20:09:39.290 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.7300999999999998e-05 2024-04-25T20:09:39.290 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.290 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:39.290 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.290 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:39.290 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:39.290 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:39.292 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:39.292+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:39.292 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T20:09:39.393 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:39.393+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:39.399 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:39.399 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:39.399 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:09:39.399 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:39.410 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T20:09:39.427 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-04-25T20:09:39.426+0000 7f249d8be640 -1 mon.a@0(leader).osd e24 definitely_dead 0 2024-04-25T20:09:39.494 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:39.494+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:39.517 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:39.517 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:39.517 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:09:39.517 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:09:39.517 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:39.525 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T20:09:39.609 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:39.609+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:39.673 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:39.673 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:09:39.673 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:09:39.673 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:09:39.673 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:39.682 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T20:09:39.696 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:39.696+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:39.797 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:39.797+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:39.827 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:39.828 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:39.828 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:09:39.828 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:39.836 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T20:09:39.898 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:39.897+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:39.941 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:39.941 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:39.942 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:09:39.942 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:09:39.942 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:39.950 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T20:09:39.999 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:39.999+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:40.008 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T20:09:40.098 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:40.099 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:09:40.099 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:09:40.099 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-04-25T20:09:40.099 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.099 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:30 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'16, mlcod=16'14)", 2024-04-25T20:09:40.099 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:09:40.099 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.930248908999999, 2024-04-25T20:09:40.099 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0054087670000000001, 2024-04-25T20:09:40.099 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:40.099 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:40.099 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:40.100 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.100 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:40.100 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:09:40.100 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.100 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.100 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:40.100 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:09:40.100 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.100 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.100 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:40.100 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167142+0000", 2024-04-25T20:09:40.100 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999976 2024-04-25T20:09:40.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.101 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.101 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:40.101 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167162+0000", 2024-04-25T20:09:40.101 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0109e-05 2024-04-25T20:09:40.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.101 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.101 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:40.101 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167188+0000", 2024-04-25T20:09:40.101 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6146e-05 2024-04-25T20:09:40.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.101 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.101 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:40.101 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167199+0000", 2024-04-25T20:09:40.101 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0767e-05 2024-04-25T20:09:40.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.102 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.102 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:40.102 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167256+0000", 2024-04-25T20:09:40.102 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7392000000000002e-05 2024-04-25T20:09:40.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.102 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.102 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:40.102 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167283+0000", 2024-04-25T20:09:40.102 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6577e-05 2024-04-25T20:09:40.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.102 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.102 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:40.102 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.172510+0000", 2024-04-25T20:09:40.103 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0052276759999999997 2024-04-25T20:09:40.103 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.103 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.103 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:40.103 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.172553+0000", 2024-04-25T20:09:40.103 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2444999999999998e-05 2024-04-25T20:09:40.103 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.103 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:40.103 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.103 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.103 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.103 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:31 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-04-25T20:09:40.103 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:09:40.103 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.929632830999999, 2024-04-25T20:09:40.103 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0055055479999999999, 2024-04-25T20:09:40.104 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:40.104 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:40.104 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:40.104 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.104 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:40.104 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:09:40.104 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.104 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.104 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:40.104 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:09:40.104 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.104 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.104 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:40.105 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:09:40.105 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:40.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.105 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.105 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:40.105 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167766+0000", 2024-04-25T20:09:40.105 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.122e-06 2024-04-25T20:09:40.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.105 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.105 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:40.105 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167776+0000", 2024-04-25T20:09:40.105 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0515e-05 2024-04-25T20:09:40.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.105 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.105 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:40.106 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167782+0000", 2024-04-25T20:09:40.106 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1149999999999999e-06 2024-04-25T20:09:40.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.106 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.106 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:40.106 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167837+0000", 2024-04-25T20:09:40.106 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4763999999999999e-05 2024-04-25T20:09:40.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.106 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.106 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:40.106 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167855+0000", 2024-04-25T20:09:40.106 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7589e-05 2024-04-25T20:09:40.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.106 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.107 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:40.107 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173221+0000", 2024-04-25T20:09:40.107 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.005365903 2024-04-25T20:09:40.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.107 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.107 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:40.107 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173266+0000", 2024-04-25T20:09:40.107 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5059999999999999e-05 2024-04-25T20:09:40.107 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.107 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:40.107 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.107 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.107 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:32 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-04-25T20:09:40.107 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:09:40.108 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.929257646, 2024-04-25T20:09:40.108 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0051558239999999998, 2024-04-25T20:09:40.108 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:40.108 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:40.108 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:40.108 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.108 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:40.108 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:09:40.108 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.108 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.108 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:40.108 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:09:40.108 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.109 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.109 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:40.109 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:09:40.109 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:40.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.109 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.109 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:40.109 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168173+0000", 2024-04-25T20:09:40.109 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7663999999999997e-05 2024-04-25T20:09:40.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.109 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.109 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:40.109 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168182+0000", 2024-04-25T20:09:40.109 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0650000000000005e-06 2024-04-25T20:09:40.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.110 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.110 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:40.110 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168188+0000", 2024-04-25T20:09:40.110 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0589999999999999e-06 2024-04-25T20:09:40.110 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.110 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.110 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:40.110 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168239+0000", 2024-04-25T20:09:40.110 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1467e-05 2024-04-25T20:09:40.110 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.110 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.110 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:40.110 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168260+0000", 2024-04-25T20:09:40.110 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0531e-05 2024-04-25T20:09:40.110 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.111 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.111 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:40.111 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173268+0000", 2024-04-25T20:09:40.111 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0050087980000000001 2024-04-25T20:09:40.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.111 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.111 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:40.111 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173291+0000", 2024-04-25T20:09:40.111 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.268e-05 2024-04-25T20:09:40.111 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.111 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:40.111 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.111 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.112 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:33 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'19, mlcod=16'14)", 2024-04-25T20:09:40.112 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:09:40.112 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.928747226999999, 2024-04-25T20:09:40.112 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046697049999999997, 2024-04-25T20:09:40.112 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:40.112 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:40.112 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:40.112 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.112 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:40.112 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:09:40.112 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.112 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.112 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:40.112 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:09:40.113 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.113 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.113 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:40.113 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168645+0000", 2024-04-25T20:09:40.113 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:40.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.113 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.113 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:40.113 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168651+0000", 2024-04-25T20:09:40.113 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1079999999999998e-06 2024-04-25T20:09:40.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.113 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.113 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:40.113 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168660+0000", 2024-04-25T20:09:40.114 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0010000000000007e-06 2024-04-25T20:09:40.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.114 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.114 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:40.114 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168666+0000", 2024-04-25T20:09:40.114 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7780000000000004e-06 2024-04-25T20:09:40.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.114 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.114 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:40.114 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168701+0000", 2024-04-25T20:09:40.114 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4490999999999999e-05 2024-04-25T20:09:40.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.114 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.114 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:40.115 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168729+0000", 2024-04-25T20:09:40.115 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8476999999999999e-05 2024-04-25T20:09:40.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.115 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.115 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:40.115 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173294+0000", 2024-04-25T20:09:40.115 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0045650730000000002 2024-04-25T20:09:40.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.115 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.115 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:40.115 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173315+0000", 2024-04-25T20:09:40.115 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1245000000000001e-05 2024-04-25T20:09:40.115 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.115 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:40.115 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.116 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.116 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:34 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'20, mlcod=16'14)", 2024-04-25T20:09:40.116 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:09:40.116 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.928248298, 2024-04-25T20:09:40.116 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041943309999999999, 2024-04-25T20:09:40.116 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:40.116 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:40.116 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:40.116 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.116 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:40.116 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:09:40.116 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.116 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:09:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169144+0000", 2024-04-25T20:09:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169150+0000", 2024-04-25T20:09:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9150000000000001e-06 2024-04-25T20:09:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:40.118 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169159+0000", 2024-04-25T20:09:40.118 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8899999999999996e-06 2024-04-25T20:09:40.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.118 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.118 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:40.118 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169169+0000", 2024-04-25T20:09:40.118 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0441e-05 2024-04-25T20:09:40.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.118 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.118 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:40.118 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169206+0000", 2024-04-25T20:09:40.118 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6816000000000001e-05 2024-04-25T20:09:40.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.118 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169234+0000", 2024-04-25T20:09:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8008999999999999e-05 2024-04-25T20:09:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173317+0000", 2024-04-25T20:09:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0040830160000000001 2024-04-25T20:09:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173339+0000", 2024-04-25T20:09:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1897000000000001e-05 2024-04-25T20:09:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.120 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.120 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:35 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'14)", 2024-04-25T20:09:40.120 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:09:40.120 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.927755979000001, 2024-04-25T20:09:40.120 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037236529999999999, 2024-04-25T20:09:40.120 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:40.120 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:40.120 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:40.120 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.120 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:40.120 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:09:40.120 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.121 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.121 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:40.121 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:09:40.121 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.121 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.121 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:40.121 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169636+0000", 2024-04-25T20:09:40.121 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:40.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.121 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.121 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:40.121 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169678+0000", 2024-04-25T20:09:40.121 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1752000000000003e-05 2024-04-25T20:09:40.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.122 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.122 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:40.122 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169688+0000", 2024-04-25T20:09:40.122 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0098000000000001e-05 2024-04-25T20:09:40.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.122 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.122 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:40.122 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169694+0000", 2024-04-25T20:09:40.122 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2349999999999996e-06 2024-04-25T20:09:40.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.122 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.122 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:40.122 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169733+0000", 2024-04-25T20:09:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8481e-05 2024-04-25T20:09:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169750+0000", 2024-04-25T20:09:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7509000000000001e-05 2024-04-25T20:09:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173340+0000", 2024-04-25T20:09:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0035898449999999999 2024-04-25T20:09:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173361+0000", 2024-04-25T20:09:40.124 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.039e-05 2024-04-25T20:09:40.124 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.124 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:40.124 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.124 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.124 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:36 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'22, mlcod=16'14)", 2024-04-25T20:09:40.124 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:09:40.124 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.927255919, 2024-04-25T20:09:40.124 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032472120000000002, 2024-04-25T20:09:40.124 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:40.124 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:40.124 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:40.124 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.124 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:40.125 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:09:40.125 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.125 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.125 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:40.125 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:09:40.125 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.125 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.125 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:40.125 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:09:40.125 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:40.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.125 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170152+0000", 2024-04-25T20:09:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5798e-05 2024-04-25T20:09:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170161+0000", 2024-04-25T20:09:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.064e-06 2024-04-25T20:09:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170167+0000", 2024-04-25T20:09:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8390000000000003e-06 2024-04-25T20:09:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.127 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.127 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:40.127 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170202+0000", 2024-04-25T20:09:40.127 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.5120999999999998e-05 2024-04-25T20:09:40.127 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.127 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.127 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:40.127 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170219+0000", 2024-04-25T20:09:40.127 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6929999999999999e-05 2024-04-25T20:09:40.127 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.127 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173362+0000", 2024-04-25T20:09:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0031427619999999999 2024-04-25T20:09:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173384+0000", 2024-04-25T20:09:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2237000000000001e-05 2024-04-25T20:09:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:40 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'26, mlcod=16'24)", 2024-04-25T20:09:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:09:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.920218006999999, 2024-04-25T20:09:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028727330000000001, 2024-04-25T20:09:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:09:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:09:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.130 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.130 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:40.130 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177173+0000", 2024-04-25T20:09:40.130 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999976 2024-04-25T20:09:40.130 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.130 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.130 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:40.130 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177186+0000", 2024-04-25T20:09:40.130 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3533000000000001e-05 2024-04-25T20:09:40.130 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.130 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.130 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:40.130 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177209+0000", 2024-04-25T20:09:40.130 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3280000000000001e-05 2024-04-25T20:09:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177220+0000", 2024-04-25T20:09:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0465000000000001e-05 2024-04-25T20:09:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177264+0000", 2024-04-25T20:09:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4413999999999999e-05 2024-04-25T20:09:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177289+0000", 2024-04-25T20:09:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4783999999999999e-05 2024-04-25T20:09:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.179990+0000", 2024-04-25T20:09:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027012939999999999 2024-04-25T20:09:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.180048+0000", 2024-04-25T20:09:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7299000000000001e-05 2024-04-25T20:09:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:67 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'41, mlcod=16'39)", 2024-04-25T20:09:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:09:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.885456515000001, 2024-04-25T20:09:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064192549999999996, 2024-04-25T20:09:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:09:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:40.134 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:09:40.134 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.134 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.134 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:40.134 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211935+0000", 2024-04-25T20:09:40.134 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999981 2024-04-25T20:09:40.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.134 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.134 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:40.134 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211948+0000", 2024-04-25T20:09:40.134 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3531e-05 2024-04-25T20:09:40.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.134 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.135 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:40.135 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211967+0000", 2024-04-25T20:09:40.135 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8326999999999999e-05 2024-04-25T20:09:40.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.135 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.135 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:40.135 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211975+0000", 2024-04-25T20:09:40.135 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8589999999999994e-06 2024-04-25T20:09:40.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.135 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.135 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:40.135 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212020+0000", 2024-04-25T20:09:40.135 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4626999999999998e-05 2024-04-25T20:09:40.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.135 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.135 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:40.136 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212045+0000", 2024-04-25T20:09:40.136 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4508999999999999e-05 2024-04-25T20:09:40.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.136 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.136 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:40.136 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218316+0000", 2024-04-25T20:09:40.136 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0062711640000000001 2024-04-25T20:09:40.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.136 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.136 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:40.136 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218356+0000", 2024-04-25T20:09:40.136 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9993000000000001e-05 2024-04-25T20:09:40.136 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.136 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:40.136 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:68 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-04-25T20:09:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:09:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.885007501, 2024-04-25T20:09:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011231365, 2024-04-25T20:09:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:09:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:09:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:09:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212392+0000", 2024-04-25T20:09:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.79e-06 2024-04-25T20:09:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:40.139 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212406+0000", 2024-04-25T20:09:40.139 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3577000000000001e-05 2024-04-25T20:09:40.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.139 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.139 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:40.139 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212414+0000", 2024-04-25T20:09:40.139 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6510000000000001e-06 2024-04-25T20:09:40.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.139 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.139 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:40.139 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217114+0000", 2024-04-25T20:09:40.139 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047005379999999998 2024-04-25T20:09:40.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.139 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217141+0000", 2024-04-25T20:09:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6984000000000001e-05 2024-04-25T20:09:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223563+0000", 2024-04-25T20:09:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064216999999999998 2024-04-25T20:09:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223617+0000", 2024-04-25T20:09:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4011000000000002e-05 2024-04-25T20:09:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:40.141 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.141 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.141 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:69 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-04-25T20:09:40.141 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:09:40.141 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.884475310999999, 2024-04-25T20:09:40.141 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.010730094000000001, 2024-04-25T20:09:40.141 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:40.141 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:40.141 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:40.141 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.141 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:40.141 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:09:40.141 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.142 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.142 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:40.142 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:09:40.142 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.142 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.142 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.142 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:40.142 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212917+0000", 2024-04-25T20:09:40.142 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:40.142 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.142 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.142 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:40.142 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212957+0000", 2024-04-25T20:09:40.142 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0692000000000003e-05 2024-04-25T20:09:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212973+0000", 2024-04-25T20:09:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.575e-05 2024-04-25T20:09:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212980+0000", 2024-04-25T20:09:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1679999999999998e-06 2024-04-25T20:09:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217449+0000", 2024-04-25T20:09:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.004468367 2024-04-25T20:09:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217470+0000", 2024-04-25T20:09:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0749000000000001e-05 2024-04-25T20:09:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223621+0000", 2024-04-25T20:09:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0061512880000000004 2024-04-25T20:09:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223648+0000", 2024-04-25T20:09:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6961000000000002e-05 2024-04-25T20:09:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:70 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'44, mlcod=16'39)", 2024-04-25T20:09:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:09:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.883716583000002, 2024-04-25T20:09:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.013553358, 2024-04-25T20:09:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:09:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:09:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:09:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213682+0000", 2024-04-25T20:09:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3439999999999997e-06 2024-04-25T20:09:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213693+0000", 2024-04-25T20:09:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1049999999999999e-05 2024-04-25T20:09:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213699+0000", 2024-04-25T20:09:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9399999999999999e-06 2024-04-25T20:09:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218086+0000", 2024-04-25T20:09:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0043871240000000001 2024-04-25T20:09:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218106+0000", 2024-04-25T20:09:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.997e-05 2024-04-25T20:09:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.227186+0000", 2024-04-25T20:09:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.009079574 2024-04-25T20:09:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.227230+0000", 2024-04-25T20:09:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4147999999999998e-05 2024-04-25T20:09:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:106 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-04-25T20:09:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:09:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.512577483000001, 2024-04-25T20:09:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002936756, 2024-04-25T20:09:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:09:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:40.150 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:09:40.150 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.150 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.150 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.150 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:40.150 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584815+0000", 2024-04-25T20:09:40.150 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:40.150 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.150 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.150 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:40.150 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584860+0000", 2024-04-25T20:09:40.150 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5161999999999999e-05 2024-04-25T20:09:40.150 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.150 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584874+0000", 2024-04-25T20:09:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4464000000000001e-05 2024-04-25T20:09:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584881+0000", 2024-04-25T20:09:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.409e-06 2024-04-25T20:09:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584915+0000", 2024-04-25T20:09:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4007000000000002e-05 2024-04-25T20:09:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584933+0000", 2024-04-25T20:09:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7737999999999999e-05 2024-04-25T20:09:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587717+0000", 2024-04-25T20:09:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002784711 2024-04-25T20:09:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587752+0000", 2024-04-25T20:09:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4915e-05 2024-04-25T20:09:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:107 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'69, mlcod=16'64)", 2024-04-25T20:09:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:09:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.512502180999999, 2024-04-25T20:09:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028906270000000002, 2024-04-25T20:09:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:09:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:09:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584890+0000", 2024-04-25T20:09:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584895+0000", 2024-04-25T20:09:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2069999999999997e-06 2024-04-25T20:09:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584906+0000", 2024-04-25T20:09:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.117e-05 2024-04-25T20:09:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584910+0000", 2024-04-25T20:09:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4190000000000004e-06 2024-04-25T20:09:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585310+0000", 2024-04-25T20:09:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00040002700000000001 2024-04-25T20:09:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585329+0000", 2024-04-25T20:09:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8838000000000002e-05 2024-04-25T20:09:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587754+0000", 2024-04-25T20:09:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024252689999999999 2024-04-25T20:09:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587781+0000", 2024-04-25T20:09:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7206999999999999e-05 2024-04-25T20:09:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.157 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:40.157 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.157 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.157 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:108 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'70, mlcod=16'64)", 2024-04-25T20:09:40.157 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:09:40.157 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.512477029999999, 2024-04-25T20:09:40.157 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028924879999999999, 2024-04-25T20:09:40.157 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:40.157 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:40.157 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:40.157 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.157 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:40.157 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:09:40.157 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:09:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:09:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584919+0000", 2024-04-25T20:09:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.557e-06 2024-04-25T20:09:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584925+0000", 2024-04-25T20:09:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8619999999999999e-06 2024-04-25T20:09:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584931+0000", 2024-04-25T20:09:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2539999999999998e-06 2024-04-25T20:09:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585550+0000", 2024-04-25T20:09:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00061821800000000002 2024-04-25T20:09:40.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.160 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.160 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:40.160 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585567+0000", 2024-04-25T20:09:40.160 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7499e-05 2024-04-25T20:09:40.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.160 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.160 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:40.160 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587783+0000", 2024-04-25T20:09:40.160 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0022156200000000002 2024-04-25T20:09:40.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.160 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.160 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:40.160 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587808+0000", 2024-04-25T20:09:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5729e-05 2024-04-25T20:09:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:112 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'74, mlcod=16'66)", 2024-04-25T20:09:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:09:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.510319408000001, 2024-04-25T20:09:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00318724, 2024-04-25T20:09:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:40.162 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:09:40.162 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.162 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.162 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:40.162 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:09:40.162 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.162 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.162 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:40.162 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587073+0000", 2024-04-25T20:09:40.162 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:40.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.163 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.163 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:40.163 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587136+0000", 2024-04-25T20:09:40.163 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2928999999999993e-05 2024-04-25T20:09:40.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.163 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.163 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:40.163 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587147+0000", 2024-04-25T20:09:40.163 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1182999999999999e-05 2024-04-25T20:09:40.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.163 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.163 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:40.163 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587153+0000", 2024-04-25T20:09:40.163 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9059999999999999e-06 2024-04-25T20:09:40.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.164 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.164 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:40.164 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587306+0000", 2024-04-25T20:09:40.164 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000152848 2024-04-25T20:09:40.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.164 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.164 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:40.164 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587330+0000", 2024-04-25T20:09:40.164 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4304e-05 2024-04-25T20:09:40.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.164 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.164 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:40.164 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.590230+0000", 2024-04-25T20:09:40.164 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029004629999999998 2024-04-25T20:09:40.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.165 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.165 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:40.165 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.590261+0000", 2024-04-25T20:09:40.165 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0323000000000001e-05 2024-04-25T20:09:40.165 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.165 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:40.165 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.165 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.165 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.165 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:118 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'80, mlcod=16'78)", 2024-04-25T20:09:40.165 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:09:40.165 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 22.505532829, 2024-04-25T20:09:40.165 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029618959999999999, 2024-04-25T20:09:40.165 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:40.166 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:40.166 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:40.166 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.166 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:40.166 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:09:40.166 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.166 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.166 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:40.166 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:09:40.166 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.166 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.166 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:40.166 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591859+0000", 2024-04-25T20:09:40.167 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:40.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.167 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.167 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:40.167 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591918+0000", 2024-04-25T20:09:40.167 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8842999999999999e-05 2024-04-25T20:09:40.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.167 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.167 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:40.167 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591933+0000", 2024-04-25T20:09:40.167 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4600000000000001e-05 2024-04-25T20:09:40.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.167 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.167 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:40.167 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591939+0000", 2024-04-25T20:09:40.168 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4559999999999997e-06 2024-04-25T20:09:40.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.168 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.168 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:40.168 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591990+0000", 2024-04-25T20:09:40.168 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0318999999999997e-05 2024-04-25T20:09:40.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.168 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.168 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:40.168 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.592008+0000", 2024-04-25T20:09:40.168 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8819000000000001e-05 2024-04-25T20:09:40.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.168 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.168 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:40.169 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.594784+0000", 2024-04-25T20:09:40.169 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027750850000000001 2024-04-25T20:09:40.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.169 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.169 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:40.169 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.594822+0000", 2024-04-25T20:09:40.169 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8522000000000001e-05 2024-04-25T20:09:40.169 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.169 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:40.169 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.169 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.169 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:139 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'97, mlcod=17'87)", 2024-04-25T20:09:40.169 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:09:40.169 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 21.971496223999999, 2024-04-25T20:09:40.170 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048546290000000001, 2024-04-25T20:09:40.170 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:40.170 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:40.170 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:40.170 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.170 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:40.170 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:09:40.170 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.170 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.170 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.170 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:40.170 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:09:40.170 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.170 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.170 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.170 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:40.171 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125896+0000", 2024-04-25T20:09:40.171 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:40.171 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.171 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.171 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:40.171 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125908+0000", 2024-04-25T20:09:40.171 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.137e-05 2024-04-25T20:09:40.171 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.171 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.171 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:40.171 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125915+0000", 2024-04-25T20:09:40.171 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3939999999999998e-06 2024-04-25T20:09:40.171 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.171 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.171 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:40.171 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125922+0000", 2024-04-25T20:09:40.172 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1930000000000004e-06 2024-04-25T20:09:40.172 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.172 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.172 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:40.172 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125959+0000", 2024-04-25T20:09:40.172 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6701e-05 2024-04-25T20:09:40.172 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.172 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.172 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:40.172 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125972+0000", 2024-04-25T20:09:40.172 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2578000000000001e-05 2024-04-25T20:09:40.172 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.172 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.172 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:40.172 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130719+0000", 2024-04-25T20:09:40.173 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047475800000000004 2024-04-25T20:09:40.173 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.173 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.173 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:40.173 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130751+0000", 2024-04-25T20:09:40.173 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2283000000000002e-05 2024-04-25T20:09:40.173 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.173 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:40.173 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.173 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.173 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.173 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:140 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'98, mlcod=17'87)", 2024-04-25T20:09:40.173 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:09:40.173 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 21.971070872999999, 2024-04-25T20:09:40.173 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048310990000000002, 2024-04-25T20:09:40.173 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:40.174 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:40.174 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:40.174 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.174 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:40.174 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:09:40.174 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.174 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.174 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.174 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:40.174 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:09:40.174 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.174 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.174 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.174 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:40.175 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:09:40.175 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:40.175 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.175 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.175 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:40.175 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126427+0000", 2024-04-25T20:09:40.175 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000105572 2024-04-25T20:09:40.175 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.175 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.175 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:40.175 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126435+0000", 2024-04-25T20:09:40.175 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1920000000000005e-06 2024-04-25T20:09:40.175 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.175 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.175 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:40.176 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126439+0000", 2024-04-25T20:09:40.176 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6950000000000002e-06 2024-04-25T20:09:40.176 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.176 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.176 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:40.176 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126471+0000", 2024-04-25T20:09:40.176 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1699000000000003e-05 2024-04-25T20:09:40.176 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.176 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.176 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:40.176 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126484+0000", 2024-04-25T20:09:40.176 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.343e-05 2024-04-25T20:09:40.176 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.176 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.176 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:40.176 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131131+0000", 2024-04-25T20:09:40.177 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046471480000000003 2024-04-25T20:09:40.177 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.177 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.177 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:40.177 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131153+0000", 2024-04-25T20:09:40.177 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1753e-05 2024-04-25T20:09:40.177 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.177 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:40.177 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.177 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.177 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.177 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:141 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'99, mlcod=17'87)", 2024-04-25T20:09:40.177 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:09:40.177 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 21.970415929000001, 2024-04-25T20:09:40.178 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.004496847, 2024-04-25T20:09:40.178 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:40.178 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:40.178 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:40.178 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.178 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:40.178 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:09:40.178 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.178 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.178 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.178 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:40.178 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:09:40.178 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:40.178 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.178 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.179 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:40.179 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:09:40.179 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:40.179 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.179 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.179 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:40.179 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126980+0000", 2024-04-25T20:09:40.179 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.619e-06 2024-04-25T20:09:40.179 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.179 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.179 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:40.179 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126987+0000", 2024-04-25T20:09:40.179 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7220000000000002e-06 2024-04-25T20:09:40.179 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.180 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.180 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:40.180 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126992+0000", 2024-04-25T20:09:40.180 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8679999999999998e-06 2024-04-25T20:09:40.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.180 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.180 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:40.180 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130772+0000", 2024-04-25T20:09:40.180 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037803120000000001 2024-04-25T20:09:40.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.180 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.180 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:40.180 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130786+0000", 2024-04-25T20:09:40.180 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3592e-05 2024-04-25T20:09:40.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.180 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.181 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:40.181 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131451+0000", 2024-04-25T20:09:40.181 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000665206 2024-04-25T20:09:40.181 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:40.181 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:40.181 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:40.181 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131474+0000", 2024-04-25T20:09:40.181 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2957999999999999e-05 2024-04-25T20:09:40.181 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.181 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:40.181 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.181 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:40.181 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:40.181 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:40.183 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T20:09:40.184 INFO:teuthology.orchestra.run.smithi039.stderr:marked out osd.3. 2024-04-25T20:09:40.184 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:09:40.184 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:09:40.185 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:09:40.185 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:09:40.186 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:40.182+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:40.193 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:09:40.194 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:09:40.210 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:09:40.210 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:09:40.227 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:09:40.228 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:09:40.261 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:09:40.262 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:09:40.284 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:40.283+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:40.286 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:09:40.286 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:09:40.303 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:09:40.303 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:09:40.318 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:40.318 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:40.327 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T20:09:40.385 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:40.384+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:40.474 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:40.474 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:40.483 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T20:09:40.485 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:40.485+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:40.587 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:40.586+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:40.588 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:40.588 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:40.597 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T20:09:40.687 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:40.687+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:40.703 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:40.703 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:40.712 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T20:09:40.796 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:40.796+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:40.858 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:40.858 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:40.867 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T20:09:40.890 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:40.890+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:40.991 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:40.991+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:41.013 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:41.013 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:41.022 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T20:09:41.093 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:41.092+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:41.137 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:41.137 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:41.146 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T20:09:41.153 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-25T20:09:41.230 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:41.229+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:41.288 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:41.288 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:41.294 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:41.293+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:41.297 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T20:09:41.394 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:41.394+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:41.438 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:41.438 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:41.446 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T20:09:41.496 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:41.495+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:41.548 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:41.548 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:41.557 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T20:09:41.596 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:41.596+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:41.697 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:41.698 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:41.698 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:41.697+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:41.706 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T20:09:41.799 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:41.798+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:41.848 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:41.848 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:41.857 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T20:09:41.900 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:41.900+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:41.960 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:41.960 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:41.968 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T20:09:42.012 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:42.011+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:42.069 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:42.069 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:42.077 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T20:09:42.102 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:42.102+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:42.163 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is set 2024-04-25T20:09:42.203 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:42.202+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:42.221 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:42.221 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:42.229 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T20:09:42.314 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:42.313+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:42.315 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T20:09:42.371 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:42.371 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:42.380 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T20:09:42.404 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:42.404+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:42.450 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:09:42.450 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:09:42.467 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:09:42.468 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:09:42.484 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:09:42.484 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:09:42.500 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:09:42.500 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:09:42.505 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:42.505+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:42.518 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:09:42.518 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:09:42.522 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:42.522 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:42.531 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T20:09:42.535 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:09:42.535 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:09:42.552 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:09:42.553 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:09:42.570 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:09:42.570 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:09:42.606 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:42.606+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:42.636 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:42.636 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:09:42.636 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:09:42.636 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [ 2024-04-25T20:09:42.637 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.637 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:09:42.637 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:09:42.637 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 25.852330937000001, 2024-04-25T20:09:42.637 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.34154860100000001, 2024-04-25T20:09:42.637 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:42.637 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:42.637 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:42.637 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:42.637 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:42.637 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 1 2024-04-25T20:09:42.637 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.637 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:42.637 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.637 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:42.638 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:09:42.638 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.638 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.638 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.638 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:42.638 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:09:42.638 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.638 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.638 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.638 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:42.638 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782261+0000", 2024-04-25T20:09:42.638 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999986 2024-04-25T20:09:42.639 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.639 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.639 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:42.639 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782268+0000", 2024-04-25T20:09:42.639 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.8140000000000001e-06 2024-04-25T20:09:42.639 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.639 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.639 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:42.639 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782279+0000", 2024-04-25T20:09:42.639 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0665999999999999e-05 2024-04-25T20:09:42.639 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.639 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.639 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:42.639 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782290+0000", 2024-04-25T20:09:42.639 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.1343e-05 2024-04-25T20:09:42.640 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.640 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.640 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:42.640 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782340+0000", 2024-04-25T20:09:42.640 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.9311999999999998e-05 2024-04-25T20:09:42.640 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.640 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.640 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for peered", 2024-04-25T20:09:42.640 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782348+0000", 2024-04-25T20:09:42.640 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.0690000000000002e-06 2024-04-25T20:09:42.640 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.640 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.640 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:42.641 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120520+0000", 2024-04-25T20:09:42.641 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.33817188100000001 2024-04-25T20:09:42.641 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.641 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.641 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:42.641 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120656+0000", 2024-04-25T20:09:42.641 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000135897 2024-04-25T20:09:42.641 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.641 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.641 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:42.641 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120802+0000", 2024-04-25T20:09:42.641 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00014691199999999999 2024-04-25T20:09:42.641 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.641 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.641 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:42.641 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.122368+0000", 2024-04-25T20:09:42.642 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0015651040000000001 2024-04-25T20:09:42.642 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.642 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.642 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:42.642 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123729+0000", 2024-04-25T20:09:42.642 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0013610499999999999 2024-04-25T20:09:42.642 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.642 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.642 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:42.642 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123769+0000", 2024-04-25T20:09:42.642 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0719000000000001e-05 2024-04-25T20:09:42.642 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.642 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.642 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:42.643 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123811+0000", 2024-04-25T20:09:42.643 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1267999999999999e-05 2024-04-25T20:09:42.643 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.643 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:42.643 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.643 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.643 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.643 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:67 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:09:42.643 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:09:42.643 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 25.422912050000001, 2024-04-25T20:09:42.643 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.006947651, 2024-04-25T20:09:42.643 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:42.643 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:42.643 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:42.643 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:42.644 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:42.644 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 67 2024-04-25T20:09:42.644 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.644 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:42.644 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.644 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:42.644 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:09:42.644 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.644 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.645 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.645 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:42.645 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:09:42.645 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.645 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.645 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.645 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:42.645 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211680+0000", 2024-04-25T20:09:42.645 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:42.645 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.645 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.645 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:42.645 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211684+0000", 2024-04-25T20:09:42.645 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4249999999999998e-06 2024-04-25T20:09:42.646 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.646 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.646 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:42.646 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211690+0000", 2024-04-25T20:09:42.646 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.7360000000000004e-06 2024-04-25T20:09:42.646 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.646 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.646 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:42.646 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211694+0000", 2024-04-25T20:09:42.646 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4860000000000002e-06 2024-04-25T20:09:42.646 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.646 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.646 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:42.647 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211727+0000", 2024-04-25T20:09:42.647 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3445999999999998e-05 2024-04-25T20:09:42.647 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.647 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.647 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:42.647 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211778+0000", 2024-04-25T20:09:42.647 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1050000000000001e-05 2024-04-25T20:09:42.647 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.647 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.647 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:42.647 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211851+0000", 2024-04-25T20:09:42.647 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2914e-05 2024-04-25T20:09:42.647 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.647 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.647 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:42.648 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213824+0000", 2024-04-25T20:09:42.648 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0019726520000000001 2024-04-25T20:09:42.648 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.648 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.648 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:42.648 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218579+0000", 2024-04-25T20:09:42.648 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0047553159999999999 2024-04-25T20:09:42.648 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.648 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.648 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:42.648 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218607+0000", 2024-04-25T20:09:42.648 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8144999999999999e-05 2024-04-25T20:09:42.648 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.648 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.648 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:42.649 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218629+0000", 2024-04-25T20:09:42.649 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.1044e-05 2024-04-25T20:09:42.649 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.649 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:42.649 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.649 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.649 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.649 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:68 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:09:42.649 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:09:42.649 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 25.422893709, 2024-04-25T20:09:42.649 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012255596000000001, 2024-04-25T20:09:42.649 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:42.649 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:42.649 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:42.650 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:42.650 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:42.650 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 68 2024-04-25T20:09:42.650 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.650 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:42.650 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.650 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:42.650 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:09:42.650 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.650 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.650 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.650 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:42.651 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:09:42.651 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.651 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.651 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.651 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:42.651 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:09:42.651 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:42.651 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.651 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.651 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:42.651 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211701+0000", 2024-04-25T20:09:42.651 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2029999999999999e-06 2024-04-25T20:09:42.652 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.652 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.652 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:42.652 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211705+0000", 2024-04-25T20:09:42.652 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.2320000000000003e-06 2024-04-25T20:09:42.652 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.652 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.652 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:42.652 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211708+0000", 2024-04-25T20:09:42.652 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.6290000000000002e-06 2024-04-25T20:09:42.652 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.652 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.652 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:42.653 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212136+0000", 2024-04-25T20:09:42.653 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00042764099999999998 2024-04-25T20:09:42.653 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.653 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.653 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:42.653 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212212+0000", 2024-04-25T20:09:42.653 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.6315000000000004e-05 2024-04-25T20:09:42.653 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.653 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.653 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:42.653 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212306+0000", 2024-04-25T20:09:42.653 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.3556999999999999e-05 2024-04-25T20:09:42.653 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.653 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.653 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:42.654 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213829+0000", 2024-04-25T20:09:42.654 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001523256 2024-04-25T20:09:42.654 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.654 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.654 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:42.654 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223904+0000", 2024-04-25T20:09:42.654 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010075387 2024-04-25T20:09:42.654 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.654 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.654 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:42.654 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223934+0000", 2024-04-25T20:09:42.654 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9988000000000002e-05 2024-04-25T20:09:42.654 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.654 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.655 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:42.655 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223955+0000", 2024-04-25T20:09:42.655 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0630999999999999e-05 2024-04-25T20:09:42.655 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.655 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:42.655 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.655 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.655 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.655 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:69 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:09:42.655 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:09:42.655 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 25.422871974, 2024-04-25T20:09:42.655 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012316863000000001, 2024-04-25T20:09:42.655 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:42.655 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:42.656 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:42.656 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:42.656 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:42.656 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 69 2024-04-25T20:09:42.656 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.656 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:42.656 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.656 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:42.656 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:09:42.656 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.656 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.656 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.656 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:42.656 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:09:42.657 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.657 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.657 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.657 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:42.657 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:09:42.657 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:42.657 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.657 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.657 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:42.657 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211762+0000", 2024-04-25T20:09:42.657 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1703999999999999e-05 2024-04-25T20:09:42.657 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.658 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.658 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:42.658 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211768+0000", 2024-04-25T20:09:42.658 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.3430000000000003e-06 2024-04-25T20:09:42.658 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.658 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.658 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:42.658 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211771+0000", 2024-04-25T20:09:42.658 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.18e-06 2024-04-25T20:09:42.658 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.658 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.658 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:42.658 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212652+0000", 2024-04-25T20:09:42.659 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00088095800000000004 2024-04-25T20:09:42.659 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.659 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.659 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:42.659 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212703+0000", 2024-04-25T20:09:42.659 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.0655999999999997e-05 2024-04-25T20:09:42.659 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.659 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.659 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:42.659 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212781+0000", 2024-04-25T20:09:42.659 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.8852999999999999e-05 2024-04-25T20:09:42.659 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.659 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.659 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:42.659 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.214131+0000", 2024-04-25T20:09:42.660 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0013496039999999999 2024-04-25T20:09:42.660 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.660 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.660 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:42.660 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223986+0000", 2024-04-25T20:09:42.660 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.009854804 2024-04-25T20:09:42.660 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.660 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.660 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:42.660 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.224017+0000", 2024-04-25T20:09:42.660 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.1068999999999997e-05 2024-04-25T20:09:42.660 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.660 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.660 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:42.661 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.224038+0000", 2024-04-25T20:09:42.661 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0965e-05 2024-04-25T20:09:42.661 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.661 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:42.661 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.661 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.661 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.661 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:70 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:09:42.661 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:09:42.661 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 25.422816893, 2024-04-25T20:09:42.661 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.015776348999999999, 2024-04-25T20:09:42.661 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:42.661 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:42.661 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:42.661 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:42.662 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:42.662 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 70 2024-04-25T20:09:42.662 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.662 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:42.662 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.662 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:42.662 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:09:42.662 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.662 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.662 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.662 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:42.662 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:09:42.662 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.662 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.663 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.663 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:42.663 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:09:42.663 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:42.663 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.663 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.663 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:42.663 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211778+0000", 2024-04-25T20:09:42.663 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5330000000000001e-06 2024-04-25T20:09:42.663 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.663 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.663 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:42.663 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211782+0000", 2024-04-25T20:09:42.663 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4079999999999998e-06 2024-04-25T20:09:42.664 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.664 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.664 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:42.664 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211786+0000", 2024-04-25T20:09:42.664 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.6190000000000002e-06 2024-04-25T20:09:42.664 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.664 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.664 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:42.664 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213374+0000", 2024-04-25T20:09:42.664 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0015881859999999999 2024-04-25T20:09:42.664 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.664 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.664 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:42.664 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213478+0000", 2024-04-25T20:09:42.665 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000103464 2024-04-25T20:09:42.665 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.665 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.665 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:42.665 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213601+0000", 2024-04-25T20:09:42.665 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00012324300000000001 2024-04-25T20:09:42.665 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.665 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.665 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:42.665 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.214788+0000", 2024-04-25T20:09:42.665 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011868569999999999 2024-04-25T20:09:42.665 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.665 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.665 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:42.665 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227504+0000", 2024-04-25T20:09:42.665 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012715749 2024-04-25T20:09:42.666 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.666 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.666 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:42.666 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227532+0000", 2024-04-25T20:09:42.666 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.7937e-05 2024-04-25T20:09:42.666 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.666 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.666 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:42.666 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227552+0000", 2024-04-25T20:09:42.666 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0772999999999999e-05 2024-04-25T20:09:42.666 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.666 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:42.666 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.667 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.667 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.667 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:131 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:42.667 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:09:42.667 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 24.513807048, 2024-04-25T20:09:42.667 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0086158810000000006, 2024-04-25T20:09:42.667 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:42.667 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:42.667 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:42.667 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:42.667 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:42.667 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 131 2024-04-25T20:09:42.667 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.667 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:42.667 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.668 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:42.668 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:09:42.668 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.668 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.668 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.668 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:42.668 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:09:42.668 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.668 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.668 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.668 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:42.668 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120785+0000", 2024-04-25T20:09:42.668 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:42.668 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.669 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.669 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:42.669 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120880+0000", 2024-04-25T20:09:42.669 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.4925000000000003e-05 2024-04-25T20:09:42.669 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.669 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.669 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:42.669 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120891+0000", 2024-04-25T20:09:42.669 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0915e-05 2024-04-25T20:09:42.669 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.669 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.669 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:42.669 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120895+0000", 2024-04-25T20:09:42.669 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.8070000000000001e-06 2024-04-25T20:09:42.669 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.670 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.670 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:42.670 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121640+0000", 2024-04-25T20:09:42.670 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00074468500000000003 2024-04-25T20:09:42.670 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.670 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.670 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:42.670 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121684+0000", 2024-04-25T20:09:42.670 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4452000000000001e-05 2024-04-25T20:09:42.670 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.670 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.670 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:42.670 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121753+0000", 2024-04-25T20:09:42.670 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8508000000000002e-05 2024-04-25T20:09:42.671 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.671 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.671 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:42.671 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123618+0000", 2024-04-25T20:09:42.671 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0018655099999999999 2024-04-25T20:09:42.671 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.671 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.671 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:42.671 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129248+0000", 2024-04-25T20:09:42.671 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0056295540000000002 2024-04-25T20:09:42.671 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.671 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.671 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:42.671 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129368+0000", 2024-04-25T20:09:42.672 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000120244 2024-04-25T20:09:42.672 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.672 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.672 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:42.672 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129402+0000", 2024-04-25T20:09:42.672 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3812000000000001e-05 2024-04-25T20:09:42.672 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.672 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:42.672 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.672 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.672 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.672 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:132 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:42.672 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:09:42.672 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 24.513692838000001, 2024-04-25T20:09:42.673 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0085898169999999996, 2024-04-25T20:09:42.673 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:42.673 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:42.673 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:42.673 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:42.673 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:42.673 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 132 2024-04-25T20:09:42.673 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.673 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:42.673 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.673 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:42.673 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:09:42.673 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.673 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.673 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.674 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:42.674 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:09:42.674 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.674 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.674 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.674 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:42.674 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:09:42.674 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:42.674 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.674 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.674 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:42.674 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120902+0000", 2024-04-25T20:09:42.674 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.6129999999999998e-06 2024-04-25T20:09:42.674 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.675 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.675 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:42.675 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120906+0000", 2024-04-25T20:09:42.675 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6569999999999999e-06 2024-04-25T20:09:42.675 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.675 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.675 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:42.675 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120909+0000", 2024-04-25T20:09:42.677 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5550000000000001e-06 2024-04-25T20:09:42.678 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.678 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.678 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:42.678 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122098+0000", 2024-04-25T20:09:42.678 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011898049999999999 2024-04-25T20:09:42.678 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.678 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.678 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:42.678 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122172+0000", 2024-04-25T20:09:42.678 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3917000000000004e-05 2024-04-25T20:09:42.678 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.678 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.678 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:42.679 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122288+0000", 2024-04-25T20:09:42.679 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00011566 2024-04-25T20:09:42.679 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.679 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.679 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:42.679 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124519+0000", 2024-04-25T20:09:42.679 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0022308359999999999 2024-04-25T20:09:42.679 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.679 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.679 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:42.679 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129414+0000", 2024-04-25T20:09:42.679 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048953720000000003 2024-04-25T20:09:42.679 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.680 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.680 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:42.680 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129458+0000", 2024-04-25T20:09:42.680 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.3426000000000001e-05 2024-04-25T20:09:42.680 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.680 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.680 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:42.680 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129490+0000", 2024-04-25T20:09:42.680 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2329999999999997e-05 2024-04-25T20:09:42.680 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.680 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:42.680 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.680 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.680 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.680 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:133 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:42.681 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:09:42.681 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 24.513638968999999, 2024-04-25T20:09:42.681 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0086066270000000004, 2024-04-25T20:09:42.681 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:42.681 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:42.681 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:42.681 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:42.681 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:42.681 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 133 2024-04-25T20:09:42.681 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.681 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:42.681 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.681 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:42.681 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:09:42.682 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.682 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.682 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.682 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:42.682 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:09:42.682 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.682 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.682 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.682 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:42.682 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:09:42.682 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:42.682 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.683 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.683 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:42.683 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120956+0000", 2024-04-25T20:09:42.683 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5289999999999998e-06 2024-04-25T20:09:42.683 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.683 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.683 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:42.683 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120962+0000", 2024-04-25T20:09:42.683 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.9490000000000001e-06 2024-04-25T20:09:42.683 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.683 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.683 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:42.683 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120965+0000", 2024-04-25T20:09:42.684 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.1860000000000001e-06 2024-04-25T20:09:42.684 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.684 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.684 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:42.684 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122666+0000", 2024-04-25T20:09:42.684 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0017004500000000001 2024-04-25T20:09:42.684 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.684 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.684 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:42.684 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122741+0000", 2024-04-25T20:09:42.684 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.5297999999999996e-05 2024-04-25T20:09:42.684 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.684 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.684 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:42.685 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122858+0000", 2024-04-25T20:09:42.685 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00011723899999999999 2024-04-25T20:09:42.685 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.685 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.685 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:42.685 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124625+0000", 2024-04-25T20:09:42.685 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0017670100000000001 2024-04-25T20:09:42.685 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.685 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.685 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:42.685 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129497+0000", 2024-04-25T20:09:42.685 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048713580000000001 2024-04-25T20:09:42.685 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.686 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.686 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:42.686 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129532+0000", 2024-04-25T20:09:42.686 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5704999999999998e-05 2024-04-25T20:09:42.686 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.686 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.686 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:42.686 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129561+0000", 2024-04-25T20:09:42.687 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8283e-05 2024-04-25T20:09:42.687 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.687 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:42.687 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.687 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.687 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.687 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:134 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:42.687 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:09:42.687 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 24.513536770999998, 2024-04-25T20:09:42.687 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0085771179999999999, 2024-04-25T20:09:42.688 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:42.688 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:42.688 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:42.688 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:42.688 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:42.688 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 134 2024-04-25T20:09:42.688 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.688 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:42.688 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.688 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:42.688 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:09:42.688 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.688 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.688 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.689 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:42.689 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:09:42.689 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.689 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.689 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.689 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:42.689 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:09:42.689 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:42.689 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.689 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.689 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:42.689 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121157+0000", 2024-04-25T20:09:42.689 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00010104599999999999 2024-04-25T20:09:42.689 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.690 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.690 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:42.690 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121189+0000", 2024-04-25T20:09:42.690 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2589000000000003e-05 2024-04-25T20:09:42.690 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.690 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.690 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:42.690 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121195+0000", 2024-04-25T20:09:42.690 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.083e-06 2024-04-25T20:09:42.690 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.690 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.690 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:42.690 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123136+0000", 2024-04-25T20:09:42.690 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0019409780000000001 2024-04-25T20:09:42.690 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.691 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.691 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:42.691 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123181+0000", 2024-04-25T20:09:42.691 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4267999999999997e-05 2024-04-25T20:09:42.691 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.691 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.691 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:42.691 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123247+0000", 2024-04-25T20:09:42.691 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.6086000000000004e-05 2024-04-25T20:09:42.691 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.691 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.691 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:42.691 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125639+0000", 2024-04-25T20:09:42.691 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.002392 2024-04-25T20:09:42.691 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.692 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.692 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:42.692 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129567+0000", 2024-04-25T20:09:42.692 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0039282170000000003 2024-04-25T20:09:42.692 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.692 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.692 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:42.692 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129605+0000", 2024-04-25T20:09:42.692 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.7660999999999997e-05 2024-04-25T20:09:42.692 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.692 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.692 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:42.692 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129633+0000", 2024-04-25T20:09:42.692 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8649999999999998e-05 2024-04-25T20:09:42.693 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.693 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:42.693 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.693 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.693 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.693 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:135 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:42.693 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:09:42.693 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 24.513377765000001, 2024-04-25T20:09:42.693 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010898453000000001, 2024-04-25T20:09:42.693 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:42.693 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:42.693 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:42.693 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:42.693 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:42.694 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 135 2024-04-25T20:09:42.694 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.694 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:42.694 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.694 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:42.694 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:09:42.694 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.694 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.694 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.694 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:42.694 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:09:42.694 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.694 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.695 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.695 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:42.695 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121214+0000", 2024-04-25T20:09:42.695 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:42.695 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.695 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.695 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:42.695 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121218+0000", 2024-04-25T20:09:42.695 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6619999999999998e-06 2024-04-25T20:09:42.695 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.695 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.695 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:42.695 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121223+0000", 2024-04-25T20:09:42.695 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1070000000000004e-06 2024-04-25T20:09:42.696 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.696 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.696 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:42.696 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121227+0000", 2024-04-25T20:09:42.696 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2820000000000001e-06 2024-04-25T20:09:42.696 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.696 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.696 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:42.696 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123649+0000", 2024-04-25T20:09:42.696 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0024221970000000001 2024-04-25T20:09:42.696 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.696 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.696 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:42.697 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123723+0000", 2024-04-25T20:09:42.697 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.4386000000000002e-05 2024-04-25T20:09:42.697 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.697 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.697 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:42.697 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123844+0000", 2024-04-25T20:09:42.697 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000121249 2024-04-25T20:09:42.697 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.697 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.697 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:42.697 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125670+0000", 2024-04-25T20:09:42.697 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001825901 2024-04-25T20:09:42.697 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.697 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.697 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:42.698 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132043+0000", 2024-04-25T20:09:42.698 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0063724539999999996 2024-04-25T20:09:42.698 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.698 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.698 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:42.698 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132084+0000", 2024-04-25T20:09:42.698 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0963000000000001e-05 2024-04-25T20:09:42.698 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.698 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.698 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:42.698 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132114+0000", 2024-04-25T20:09:42.698 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9927e-05 2024-04-25T20:09:42.698 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.699 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:42.699 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.699 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.699 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.699 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:136 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:42.699 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:09:42.699 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 24.513362238999999, 2024-04-25T20:09:42.699 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010952513000000001, 2024-04-25T20:09:42.699 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:42.699 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:42.699 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:42.699 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:42.699 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:42.699 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 136 2024-04-25T20:09:42.700 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.700 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:42.700 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.700 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:42.700 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:09:42.700 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.700 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.700 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.700 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:42.700 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:09:42.700 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.700 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.700 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.701 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:42.701 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121230+0000", 2024-04-25T20:09:42.701 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:42.701 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.701 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.701 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:42.701 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121236+0000", 2024-04-25T20:09:42.701 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.7320000000000003e-06 2024-04-25T20:09:42.701 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.702 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.702 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:42.702 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121242+0000", 2024-04-25T20:09:42.702 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.7810000000000002e-06 2024-04-25T20:09:42.702 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.702 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.703 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:42.703 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121246+0000", 2024-04-25T20:09:42.703 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.639e-06 2024-04-25T20:09:42.703 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.703 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.703 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:42.703 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124150+0000", 2024-04-25T20:09:42.704 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.002903816 2024-04-25T20:09:42.704 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.704 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.704 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:42.704 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124195+0000", 2024-04-25T20:09:42.704 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.5042e-05 2024-04-25T20:09:42.704 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.704 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.704 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:42.704 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124266+0000", 2024-04-25T20:09:42.704 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0234999999999994e-05 2024-04-25T20:09:42.704 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.704 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.705 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:42.705 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125711+0000", 2024-04-25T20:09:42.705 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0014458800000000001 2024-04-25T20:09:42.705 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.705 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.705 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:42.705 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132120+0000", 2024-04-25T20:09:42.705 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0064084399999999996 2024-04-25T20:09:42.705 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.705 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.705 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:42.706 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132155+0000", 2024-04-25T20:09:42.706 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4938000000000003e-05 2024-04-25T20:09:42.706 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.706 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.706 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:42.706 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132183+0000", 2024-04-25T20:09:42.706 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8360999999999998e-05 2024-04-25T20:09:42.707 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.707 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:42.707 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.707 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.707 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.707 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:42.707 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:09:42.707 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 24.513312506999998, 2024-04-25T20:09:42.707 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.01097684, 2024-04-25T20:09:42.707 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:42.707 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:42.707 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:42.707 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:42.707 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:42.708 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 137 2024-04-25T20:09:42.708 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.708 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:42.708 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.708 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:42.708 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:09:42.708 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.708 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.708 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.708 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:42.708 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:09:42.708 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.708 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.708 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.709 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:42.709 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:09:42.709 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:42.709 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.709 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.709 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:42.709 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121328+0000", 2024-04-25T20:09:42.709 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.7908999999999998e-05 2024-04-25T20:09:42.709 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.709 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.709 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:42.709 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121337+0000", 2024-04-25T20:09:42.709 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.9369999999999993e-06 2024-04-25T20:09:42.709 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.710 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.710 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:42.710 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121343+0000", 2024-04-25T20:09:42.710 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.8089999999999998e-06 2024-04-25T20:09:42.710 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.710 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.710 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:42.710 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124656+0000", 2024-04-25T20:09:42.710 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0033133839999999999 2024-04-25T20:09:42.710 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.710 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.710 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:42.711 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124730+0000", 2024-04-25T20:09:42.711 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3565000000000005e-05 2024-04-25T20:09:42.711 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.711 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.711 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:42.711 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124846+0000", 2024-04-25T20:09:42.711 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000116416 2024-04-25T20:09:42.711 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.711 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.711 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:42.711 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126756+0000", 2024-04-25T20:09:42.711 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001909991 2024-04-25T20:09:42.711 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.711 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.711 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:42.712 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132190+0000", 2024-04-25T20:09:42.712 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0054343880000000001 2024-04-25T20:09:42.712 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.712 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.712 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:42.712 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132227+0000", 2024-04-25T20:09:42.712 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6875000000000002e-05 2024-04-25T20:09:42.712 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.712 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.712 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:42.712 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132257+0000", 2024-04-25T20:09:42.712 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.0046e-05 2024-04-25T20:09:42.712 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.713 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:42.713 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.713 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.713 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.713 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:42.713 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:09:42.713 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 24.513242549000001, 2024-04-25T20:09:42.713 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010978653999999999, 2024-04-25T20:09:42.714 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:42.714 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:42.714 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:42.714 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:42.714 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:42.714 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 138 2024-04-25T20:09:42.715 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.715 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:42.715 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.715 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:42.715 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:09:42.715 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.715 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.716 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.716 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:42.716 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:09:42.716 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.716 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.716 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.716 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:42.716 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:09:42.716 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:42.716 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.716 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.716 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:42.717 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121355+0000", 2024-04-25T20:09:42.717 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.9869999999999999e-06 2024-04-25T20:09:42.717 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.717 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.717 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:42.717 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121361+0000", 2024-04-25T20:09:42.717 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.4620000000000001e-06 2024-04-25T20:09:42.717 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.717 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.717 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:42.717 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121373+0000", 2024-04-25T20:09:42.717 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.2035000000000001e-05 2024-04-25T20:09:42.718 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.718 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.718 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:42.718 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125267+0000", 2024-04-25T20:09:42.718 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0038939230000000001 2024-04-25T20:09:42.718 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.718 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.718 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:42.718 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125309+0000", 2024-04-25T20:09:42.718 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1690999999999998e-05 2024-04-25T20:09:42.718 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.718 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.718 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:42.719 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125385+0000", 2024-04-25T20:09:42.719 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.6162999999999996e-05 2024-04-25T20:09:42.719 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.719 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.719 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:42.719 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126798+0000", 2024-04-25T20:09:42.719 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0014124680000000001 2024-04-25T20:09:42.719 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.719 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.719 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:42.719 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132264+0000", 2024-04-25T20:09:42.719 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0054661689999999999 2024-04-25T20:09:42.719 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.720 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.720 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:42.720 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132299+0000", 2024-04-25T20:09:42.720 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5272999999999999e-05 2024-04-25T20:09:42.720 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.720 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.720 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:42.720 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132329+0000", 2024-04-25T20:09:42.720 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.0080999999999999e-05 2024-04-25T20:09:42.720 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.720 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:42.720 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.720 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.720 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.720 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:42.721 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:09:42.721 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 24.513211974000001, 2024-04-25T20:09:42.721 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011015853000000001, 2024-04-25T20:09:42.721 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:42.721 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:42.721 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:42.721 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:42.721 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:42.721 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 139 2024-04-25T20:09:42.721 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.721 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:42.721 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.721 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:42.721 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:09:42.722 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.722 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.722 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.722 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:42.722 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:09:42.722 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.722 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.722 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.722 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:42.722 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121380+0000", 2024-04-25T20:09:42.722 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:42.722 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.722 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.722 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:42.723 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121385+0000", 2024-04-25T20:09:42.723 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.092e-06 2024-04-25T20:09:42.723 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.723 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.723 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:42.723 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121390+0000", 2024-04-25T20:09:42.723 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.5990000000000001e-06 2024-04-25T20:09:42.723 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.723 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.723 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:42.723 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121397+0000", 2024-04-25T20:09:42.723 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.6279999999999999e-06 2024-04-25T20:09:42.723 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.724 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.724 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:42.724 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125737+0000", 2024-04-25T20:09:42.724 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0043406010000000004 2024-04-25T20:09:42.724 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.724 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.724 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:42.724 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125781+0000", 2024-04-25T20:09:42.724 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.3958999999999997e-05 2024-04-25T20:09:42.724 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.724 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.724 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:42.724 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125849+0000", 2024-04-25T20:09:42.724 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8071999999999995e-05 2024-04-25T20:09:42.725 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.725 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.725 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:42.725 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.130908+0000", 2024-04-25T20:09:42.725 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0050586509999999999 2024-04-25T20:09:42.725 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.725 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.725 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:42.725 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132334+0000", 2024-04-25T20:09:42.725 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00142621 2024-04-25T20:09:42.725 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.725 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.725 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:42.725 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132369+0000", 2024-04-25T20:09:42.725 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4452000000000002e-05 2024-04-25T20:09:42.726 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.726 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.726 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:42.726 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132397+0000", 2024-04-25T20:09:42.726 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8065e-05 2024-04-25T20:09:42.726 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.726 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:42.726 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.726 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.726 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.726 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:42.726 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:09:42.726 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 24.51319093, 2024-04-25T20:09:42.727 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011069443, 2024-04-25T20:09:42.727 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:42.727 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:42.727 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:42.727 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:42.727 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:42.727 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 140 2024-04-25T20:09:42.727 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.727 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:42.727 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.727 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:42.727 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:09:42.727 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.727 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.727 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.728 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:42.728 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:09:42.728 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.728 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.728 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.728 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:42.728 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:09:42.728 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:42.728 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.728 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.728 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:42.728 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121456+0000", 2024-04-25T20:09:42.728 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.4469000000000003e-05 2024-04-25T20:09:42.728 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.728 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.729 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:42.729 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121462+0000", 2024-04-25T20:09:42.729 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.1890000000000004e-06 2024-04-25T20:09:42.729 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.729 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.729 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:42.729 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121466+0000", 2024-04-25T20:09:42.729 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3909999999999998e-06 2024-04-25T20:09:42.729 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.729 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.729 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:42.729 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126071+0000", 2024-04-25T20:09:42.729 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0046048979999999996 2024-04-25T20:09:42.729 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.730 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.730 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:42.730 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126141+0000", 2024-04-25T20:09:42.730 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0621e-05 2024-04-25T20:09:42.730 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.730 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.730 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:42.730 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126260+0000", 2024-04-25T20:09:42.730 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000118621 2024-04-25T20:09:42.730 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.730 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.730 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:42.730 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.131270+0000", 2024-04-25T20:09:42.731 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0050096949999999998 2024-04-25T20:09:42.731 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.731 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.731 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:42.731 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132403+0000", 2024-04-25T20:09:42.731 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011332779999999999 2024-04-25T20:09:42.731 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.731 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.731 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:42.731 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132443+0000", 2024-04-25T20:09:42.731 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0175000000000001e-05 2024-04-25T20:09:42.731 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.731 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.731 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:42.731 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132471+0000", 2024-04-25T20:09:42.732 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8399e-05 2024-04-25T20:09:42.732 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.732 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:42.732 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.732 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.732 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.732 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:42.732 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:09:42.732 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 24.513122145000001, 2024-04-25T20:09:42.732 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011071002, 2024-04-25T20:09:42.732 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:42.732 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:42.732 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:42.732 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:42.733 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:42.733 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 141 2024-04-25T20:09:42.733 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.733 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:42.733 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.733 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:42.733 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:09:42.733 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.733 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.733 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.733 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:42.733 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:09:42.733 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.734 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.734 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.734 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:42.734 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121470+0000", 2024-04-25T20:09:42.734 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:42.734 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.734 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.734 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:42.734 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121473+0000", 2024-04-25T20:09:42.734 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.01e-06 2024-04-25T20:09:42.734 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.734 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.734 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:42.734 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121478+0000", 2024-04-25T20:09:42.735 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.515e-06 2024-04-25T20:09:42.735 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.735 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.735 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:42.735 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121480+0000", 2024-04-25T20:09:42.735 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5749999999999999e-06 2024-04-25T20:09:42.735 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.735 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.735 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:42.735 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126813+0000", 2024-04-25T20:09:42.735 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0053320399999999997 2024-04-25T20:09:42.735 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.735 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.735 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:42.735 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126854+0000", 2024-04-25T20:09:42.736 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1582999999999998e-05 2024-04-25T20:09:42.736 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.736 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.736 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:42.736 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126923+0000", 2024-04-25T20:09:42.736 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8544e-05 2024-04-25T20:09:42.736 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.736 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.736 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:42.736 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.131573+0000", 2024-04-25T20:09:42.736 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0046504659999999998 2024-04-25T20:09:42.736 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.736 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.736 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:42.736 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132478+0000", 2024-04-25T20:09:42.737 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00090442400000000003 2024-04-25T20:09:42.737 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.737 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.737 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:42.737 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132513+0000", 2024-04-25T20:09:42.737 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5204999999999999e-05 2024-04-25T20:09:42.737 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.737 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.737 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:42.737 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132542+0000", 2024-04-25T20:09:42.737 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.904e-05 2024-04-25T20:09:42.737 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.737 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:42.737 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.737 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.737 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.738 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:156 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:42.738 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:09:42.738 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 24.01529785, 2024-04-25T20:09:42.738 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090072280000000008, 2024-04-25T20:09:42.738 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:42.738 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:42.738 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:42.738 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:42.738 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:42.738 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 156 2024-04-25T20:09:42.738 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.738 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:42.739 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.739 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:42.739 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:09:42.739 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.739 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.739 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.739 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:42.739 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:09:42.739 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.739 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.739 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.739 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:42.739 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:09:42.739 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:42.740 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.740 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.740 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:42.740 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619297+0000", 2024-04-25T20:09:42.740 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2979999999999999e-06 2024-04-25T20:09:42.740 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.740 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.740 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:42.740 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619300+0000", 2024-04-25T20:09:42.740 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.371e-06 2024-04-25T20:09:42.740 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.740 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.740 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:42.740 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619304+0000", 2024-04-25T20:09:42.740 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.304e-06 2024-04-25T20:09:42.741 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.741 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.741 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:42.741 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.622910+0000", 2024-04-25T20:09:42.741 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0036065049999999999 2024-04-25T20:09:42.741 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.741 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.741 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:42.741 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.622990+0000", 2024-04-25T20:09:42.741 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.9722999999999995e-05 2024-04-25T20:09:42.741 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.741 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.741 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:42.741 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623128+0000", 2024-04-25T20:09:42.741 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00013818399999999999 2024-04-25T20:09:42.742 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.742 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.742 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:42.742 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625155+0000", 2024-04-25T20:09:42.742 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0020267979999999998 2024-04-25T20:09:42.742 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.742 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.742 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:42.742 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628225+0000", 2024-04-25T20:09:42.742 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0030705680000000001 2024-04-25T20:09:42.742 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.742 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.742 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:42.743 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628268+0000", 2024-04-25T20:09:42.743 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.2404000000000003e-05 2024-04-25T20:09:42.743 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.743 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.743 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:42.743 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628302+0000", 2024-04-25T20:09:42.743 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4439e-05 2024-04-25T20:09:42.743 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.743 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:42.743 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.743 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.743 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.743 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:157 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:42.743 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:09:42.743 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 24.015285751, 2024-04-25T20:09:42.744 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090534889999999996, 2024-04-25T20:09:42.744 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:42.744 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:42.744 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:42.744 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:42.744 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:42.744 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 157 2024-04-25T20:09:42.744 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.744 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:42.744 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.744 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:42.744 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:09:42.744 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.744 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.745 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.745 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:42.745 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:09:42.745 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.745 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.745 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.745 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:42.745 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:09:42.745 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:42.745 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.745 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.745 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:42.746 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619309+0000", 2024-04-25T20:09:42.746 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.841e-06 2024-04-25T20:09:42.746 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.746 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.746 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:42.746 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619312+0000", 2024-04-25T20:09:42.746 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.084e-06 2024-04-25T20:09:42.746 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.746 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.746 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:42.746 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619314+0000", 2024-04-25T20:09:42.746 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.3860000000000001e-06 2024-04-25T20:09:42.746 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.747 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.747 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:42.747 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623718+0000", 2024-04-25T20:09:42.747 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0044038050000000002 2024-04-25T20:09:42.747 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.747 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.747 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:42.747 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623791+0000", 2024-04-25T20:09:42.747 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3181000000000003e-05 2024-04-25T20:09:42.747 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.747 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.747 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:42.747 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623922+0000", 2024-04-25T20:09:42.747 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00013117000000000001 2024-04-25T20:09:42.747 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.748 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.748 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:42.748 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625187+0000", 2024-04-25T20:09:42.748 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001264961 2024-04-25T20:09:42.748 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.748 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.748 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:42.748 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628308+0000", 2024-04-25T20:09:42.748 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.003120739 2024-04-25T20:09:42.748 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.748 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.748 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:42.748 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628331+0000", 2024-04-25T20:09:42.749 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2802999999999998e-05 2024-04-25T20:09:42.749 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.749 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.749 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:42.749 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628361+0000", 2024-04-25T20:09:42.749 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9760999999999998e-05 2024-04-25T20:09:42.749 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.749 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:42.749 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.749 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.749 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.749 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:158 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:42.749 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:09:42.749 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 24.015275303999999, 2024-04-25T20:09:42.750 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0091076409999999997, 2024-04-25T20:09:42.750 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:42.750 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:42.750 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:42.750 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:42.750 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:42.750 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 158 2024-04-25T20:09:42.750 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.750 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:42.750 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.750 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:42.750 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:09:42.750 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.751 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.751 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.751 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:42.751 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:09:42.751 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.751 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.751 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.751 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:42.751 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619317+0000", 2024-04-25T20:09:42.751 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967296 2024-04-25T20:09:42.751 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.752 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.752 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:42.752 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619364+0000", 2024-04-25T20:09:42.752 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.6572000000000001e-05 2024-04-25T20:09:42.752 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.752 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.752 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:42.752 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619371+0000", 2024-04-25T20:09:42.752 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3860000000000001e-06 2024-04-25T20:09:42.752 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.752 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.752 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:42.752 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619382+0000", 2024-04-25T20:09:42.752 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0173999999999999e-05 2024-04-25T20:09:42.752 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.753 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.753 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:42.753 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624216+0000", 2024-04-25T20:09:42.753 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048340350000000004 2024-04-25T20:09:42.753 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.753 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.753 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:42.753 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624260+0000", 2024-04-25T20:09:42.753 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4558999999999998e-05 2024-04-25T20:09:42.753 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.753 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.753 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:42.753 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624333+0000", 2024-04-25T20:09:42.753 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2744999999999996e-05 2024-04-25T20:09:42.754 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.754 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.754 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:42.754 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625465+0000", 2024-04-25T20:09:42.754 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001132454 2024-04-25T20:09:42.754 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.754 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.754 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:42.754 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628365+0000", 2024-04-25T20:09:42.754 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0028994950000000002 2024-04-25T20:09:42.754 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.754 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.754 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:42.755 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628386+0000", 2024-04-25T20:09:42.755 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.1367000000000001e-05 2024-04-25T20:09:42.755 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.755 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.755 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:42.755 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628425+0000", 2024-04-25T20:09:42.755 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.9087e-05 2024-04-25T20:09:42.755 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.755 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:42.755 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.755 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.755 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.755 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:159 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:42.756 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:09:42.756 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 24.015206651, 2024-04-25T20:09:42.756 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090948999999999995, 2024-04-25T20:09:42.756 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:42.756 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:42.756 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:42.756 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:42.756 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:42.756 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 159 2024-04-25T20:09:42.756 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.756 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:42.756 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.756 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:42.756 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:09:42.756 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.757 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.757 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.757 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:42.757 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:09:42.757 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:42.757 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.757 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.757 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:42.757 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:09:42.757 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:42.757 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.757 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.757 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:42.757 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619389+0000", 2024-04-25T20:09:42.758 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.734e-06 2024-04-25T20:09:42.758 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.758 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.758 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:42.758 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619392+0000", 2024-04-25T20:09:42.758 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6219999999999998e-06 2024-04-25T20:09:42.758 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.758 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.758 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:42.758 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619403+0000", 2024-04-25T20:09:42.758 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0987e-05 2024-04-25T20:09:42.758 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.759 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.759 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:42.759 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624667+0000", 2024-04-25T20:09:42.759 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0052633970000000004 2024-04-25T20:09:42.759 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.759 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.759 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:42.759 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624741+0000", 2024-04-25T20:09:42.759 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.4583000000000001e-05 2024-04-25T20:09:42.759 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.759 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.759 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:42.759 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624857+0000", 2024-04-25T20:09:42.759 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000116118 2024-04-25T20:09:42.760 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.760 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.760 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:42.760 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.626013+0000", 2024-04-25T20:09:42.760 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001155473 2024-04-25T20:09:42.760 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.760 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.760 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:42.760 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628432+0000", 2024-04-25T20:09:42.760 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0024189379999999998 2024-04-25T20:09:42.760 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.760 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.760 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:42.760 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628454+0000", 2024-04-25T20:09:42.760 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2059999999999999e-05 2024-04-25T20:09:42.761 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:42.761 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:42.761 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:42.761 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628481+0000", 2024-04-25T20:09:42.761 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.7300999999999998e-05 2024-04-25T20:09:42.761 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.761 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:42.761 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.761 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:42.761 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:42.761 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:42.763 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:42.762+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:42.763 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T20:09:42.864 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:42.863+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:42.870 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:42.870 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:42.870 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:09:42.870 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:42.882 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T20:09:42.966 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:42.966+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:43.028 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:43.028 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:43.028 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:09:43.028 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:09:43.028 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:43.037 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T20:09:43.077 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:43.077+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:43.167 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:43.167+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:43.182 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:43.182 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:09:43.182 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:09:43.182 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:09:43.182 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:43.191 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T20:09:43.275 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:43.275+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:43.338 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:43.338 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:43.338 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:09:43.338 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:43.347 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T20:09:43.369 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:43.369+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:43.470 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:43.470+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:43.494 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:43.494 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:43.494 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:09:43.494 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:09:43.494 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:43.502 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T20:09:43.571 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:43.570+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:43.650 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:43.650 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:09:43.650 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:09:43.650 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-04-25T20:09:43.650 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.650 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:30 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'16, mlcod=16'14)", 2024-04-25T20:09:43.650 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:09:43.650 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 26.481651849999999, 2024-04-25T20:09:43.650 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0054087670000000001, 2024-04-25T20:09:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:09:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:09:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:43.651 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167142+0000", 2024-04-25T20:09:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999976 2024-04-25T20:09:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167162+0000", 2024-04-25T20:09:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0109e-05 2024-04-25T20:09:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167188+0000", 2024-04-25T20:09:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6146e-05 2024-04-25T20:09:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:43.652 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167199+0000", 2024-04-25T20:09:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0767e-05 2024-04-25T20:09:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167256+0000", 2024-04-25T20:09:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7392000000000002e-05 2024-04-25T20:09:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167283+0000", 2024-04-25T20:09:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6577e-05 2024-04-25T20:09:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.172510+0000", 2024-04-25T20:09:43.653 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0052276759999999997 2024-04-25T20:09:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.172553+0000", 2024-04-25T20:09:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2444999999999998e-05 2024-04-25T20:09:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:31 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-04-25T20:09:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:09:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 26.481035771999998, 2024-04-25T20:09:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0055055479999999999, 2024-04-25T20:09:43.654 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:09:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:09:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:09:43.655 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167766+0000", 2024-04-25T20:09:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.122e-06 2024-04-25T20:09:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167776+0000", 2024-04-25T20:09:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0515e-05 2024-04-25T20:09:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167782+0000", 2024-04-25T20:09:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1149999999999999e-06 2024-04-25T20:09:43.656 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167837+0000", 2024-04-25T20:09:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4763999999999999e-05 2024-04-25T20:09:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167855+0000", 2024-04-25T20:09:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7589e-05 2024-04-25T20:09:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173221+0000", 2024-04-25T20:09:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.005365903 2024-04-25T20:09:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.657 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173266+0000", 2024-04-25T20:09:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5059999999999999e-05 2024-04-25T20:09:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:32 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-04-25T20:09:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:09:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 26.480660586999999, 2024-04-25T20:09:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0051558239999999998, 2024-04-25T20:09:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:43.658 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:43.659 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:43.659 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.659 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:43.659 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:09:43.659 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.659 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.659 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.659 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:43.659 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:09:43.659 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.659 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.659 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.659 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:43.660 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:09:43.660 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:43.660 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.660 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.660 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:43.660 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168173+0000", 2024-04-25T20:09:43.660 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7663999999999997e-05 2024-04-25T20:09:43.660 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.660 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.660 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:43.660 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168182+0000", 2024-04-25T20:09:43.660 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0650000000000005e-06 2024-04-25T20:09:43.660 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.660 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.660 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:43.661 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168188+0000", 2024-04-25T20:09:43.661 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0589999999999999e-06 2024-04-25T20:09:43.661 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.661 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.661 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:43.661 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168239+0000", 2024-04-25T20:09:43.661 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1467e-05 2024-04-25T20:09:43.661 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.661 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.661 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:43.661 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168260+0000", 2024-04-25T20:09:43.661 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0531e-05 2024-04-25T20:09:43.661 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.662 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.662 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:43.662 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173268+0000", 2024-04-25T20:09:43.662 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0050087980000000001 2024-04-25T20:09:43.662 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.662 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.662 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:43.662 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173291+0000", 2024-04-25T20:09:43.662 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.268e-05 2024-04-25T20:09:43.662 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.662 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:43.662 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.662 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.662 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.663 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:33 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'19, mlcod=16'14)", 2024-04-25T20:09:43.663 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:09:43.663 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 26.480150168000002, 2024-04-25T20:09:43.663 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046697049999999997, 2024-04-25T20:09:43.663 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:43.663 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:43.663 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:43.663 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.663 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:43.663 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:09:43.663 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.663 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.663 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.663 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:43.664 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:09:43.664 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.664 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.664 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.664 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:43.664 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168645+0000", 2024-04-25T20:09:43.664 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:43.664 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.664 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.664 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:43.664 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168651+0000", 2024-04-25T20:09:43.664 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1079999999999998e-06 2024-04-25T20:09:43.664 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.664 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.664 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:43.665 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168660+0000", 2024-04-25T20:09:43.665 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0010000000000007e-06 2024-04-25T20:09:43.665 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.665 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.665 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:43.665 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168666+0000", 2024-04-25T20:09:43.665 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7780000000000004e-06 2024-04-25T20:09:43.665 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.665 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.665 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:43.665 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168701+0000", 2024-04-25T20:09:43.665 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4490999999999999e-05 2024-04-25T20:09:43.665 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.665 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.665 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:43.665 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168729+0000", 2024-04-25T20:09:43.665 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8476999999999999e-05 2024-04-25T20:09:43.666 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.666 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.666 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:43.666 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173294+0000", 2024-04-25T20:09:43.666 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0045650730000000002 2024-04-25T20:09:43.666 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.666 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.666 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:43.666 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173315+0000", 2024-04-25T20:09:43.666 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1245000000000001e-05 2024-04-25T20:09:43.666 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.666 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:43.666 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.666 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.667 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.667 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:34 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'20, mlcod=16'14)", 2024-04-25T20:09:43.667 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:09:43.667 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 26.479651238999999, 2024-04-25T20:09:43.667 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041943309999999999, 2024-04-25T20:09:43.667 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:43.667 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:43.667 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:43.667 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.667 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:43.667 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:09:43.667 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.667 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.667 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.667 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:43.667 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:09:43.667 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.668 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.668 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.668 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:43.668 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169144+0000", 2024-04-25T20:09:43.668 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:43.668 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.668 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.668 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:43.668 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169150+0000", 2024-04-25T20:09:43.668 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9150000000000001e-06 2024-04-25T20:09:43.668 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.668 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.668 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:43.669 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169159+0000", 2024-04-25T20:09:43.669 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8899999999999996e-06 2024-04-25T20:09:43.669 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.669 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.669 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:43.669 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169169+0000", 2024-04-25T20:09:43.669 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0441e-05 2024-04-25T20:09:43.669 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.669 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.669 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:43.669 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169206+0000", 2024-04-25T20:09:43.669 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6816000000000001e-05 2024-04-25T20:09:43.669 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.669 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.669 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:43.669 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169234+0000", 2024-04-25T20:09:43.670 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8008999999999999e-05 2024-04-25T20:09:43.670 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.670 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.670 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:43.670 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173317+0000", 2024-04-25T20:09:43.670 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0040830160000000001 2024-04-25T20:09:43.670 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.670 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.670 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:43.670 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173339+0000", 2024-04-25T20:09:43.670 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1897000000000001e-05 2024-04-25T20:09:43.670 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.670 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:43.670 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.671 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.671 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.671 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:35 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'14)", 2024-04-25T20:09:43.671 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:09:43.671 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 26.47915892, 2024-04-25T20:09:43.671 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037236529999999999, 2024-04-25T20:09:43.671 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:43.671 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:43.671 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:43.671 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.671 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:43.671 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:09:43.671 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.671 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.671 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.671 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:43.671 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:09:43.672 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.672 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.672 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.672 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:43.672 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169636+0000", 2024-04-25T20:09:43.672 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:43.672 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.672 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.672 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:43.672 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169678+0000", 2024-04-25T20:09:43.672 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1752000000000003e-05 2024-04-25T20:09:43.672 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.672 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.672 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:43.672 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169688+0000", 2024-04-25T20:09:43.672 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0098000000000001e-05 2024-04-25T20:09:43.673 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.673 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.673 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:43.673 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169694+0000", 2024-04-25T20:09:43.673 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2349999999999996e-06 2024-04-25T20:09:43.673 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.673 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.673 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:43.673 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169733+0000", 2024-04-25T20:09:43.673 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8481e-05 2024-04-25T20:09:43.673 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.673 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.673 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:43.673 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169750+0000", 2024-04-25T20:09:43.673 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7509000000000001e-05 2024-04-25T20:09:43.673 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.674 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.674 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:43.674 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173340+0000", 2024-04-25T20:09:43.674 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0035898449999999999 2024-04-25T20:09:43.674 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.674 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.674 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:43.674 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173361+0000", 2024-04-25T20:09:43.674 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.039e-05 2024-04-25T20:09:43.674 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.674 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:43.674 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.674 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.674 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.675 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:36 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'22, mlcod=16'14)", 2024-04-25T20:09:43.675 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:09:43.675 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 26.478658859999999, 2024-04-25T20:09:43.675 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032472120000000002, 2024-04-25T20:09:43.675 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:43.675 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:43.675 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:43.675 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.675 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:43.675 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:09:43.675 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.675 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.675 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.675 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:43.675 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:09:43.676 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.676 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.676 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.676 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:43.676 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:09:43.676 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:43.676 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.676 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.676 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:43.676 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170152+0000", 2024-04-25T20:09:43.676 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5798e-05 2024-04-25T20:09:43.676 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.676 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.676 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:43.676 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170161+0000", 2024-04-25T20:09:43.677 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.064e-06 2024-04-25T20:09:43.677 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.677 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.677 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:43.677 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170167+0000", 2024-04-25T20:09:43.677 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8390000000000003e-06 2024-04-25T20:09:43.677 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.677 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.677 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:43.677 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170202+0000", 2024-04-25T20:09:43.677 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.5120999999999998e-05 2024-04-25T20:09:43.677 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.677 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.677 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:43.677 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170219+0000", 2024-04-25T20:09:43.677 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6929999999999999e-05 2024-04-25T20:09:43.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.678 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.678 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:43.678 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173362+0000", 2024-04-25T20:09:43.678 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0031427619999999999 2024-04-25T20:09:43.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.678 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.678 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:43.678 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173384+0000", 2024-04-25T20:09:43.678 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2237000000000001e-05 2024-04-25T20:09:43.678 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.678 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:43.678 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.678 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.679 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:40 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'26, mlcod=16'24)", 2024-04-25T20:09:43.679 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:09:43.679 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 26.471620948000002, 2024-04-25T20:09:43.679 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028727330000000001, 2024-04-25T20:09:43.679 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:43.679 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:43.679 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:43.679 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.679 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:43.679 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:09:43.679 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.679 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.679 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.679 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:43.680 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:09:43.680 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.680 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.680 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.680 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:43.680 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177173+0000", 2024-04-25T20:09:43.680 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999976 2024-04-25T20:09:43.680 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.680 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.680 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:43.680 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177186+0000", 2024-04-25T20:09:43.680 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3533000000000001e-05 2024-04-25T20:09:43.680 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.680 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.680 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:43.680 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177209+0000", 2024-04-25T20:09:43.681 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3280000000000001e-05 2024-04-25T20:09:43.681 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.681 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.681 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:43.681 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177220+0000", 2024-04-25T20:09:43.681 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0465000000000001e-05 2024-04-25T20:09:43.681 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.681 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.681 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:43.681 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177264+0000", 2024-04-25T20:09:43.681 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4413999999999999e-05 2024-04-25T20:09:43.681 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.681 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.681 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:43.681 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177289+0000", 2024-04-25T20:09:43.682 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4783999999999999e-05 2024-04-25T20:09:43.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.682 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.682 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:43.682 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.179990+0000", 2024-04-25T20:09:43.682 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027012939999999999 2024-04-25T20:09:43.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.682 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.682 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:43.682 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.180048+0000", 2024-04-25T20:09:43.682 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7299000000000001e-05 2024-04-25T20:09:43.682 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.682 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:43.682 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.683 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.683 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:67 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'41, mlcod=16'39)", 2024-04-25T20:09:43.683 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:09:43.683 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 26.436859456000001, 2024-04-25T20:09:43.683 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064192549999999996, 2024-04-25T20:09:43.683 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:43.683 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:43.683 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:43.683 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.683 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:43.683 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:09:43.683 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.683 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.683 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:43.683 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:09:43.684 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.684 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.684 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:43.684 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211935+0000", 2024-04-25T20:09:43.684 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999981 2024-04-25T20:09:43.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.684 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.684 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:43.684 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211948+0000", 2024-04-25T20:09:43.684 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3531e-05 2024-04-25T20:09:43.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.684 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.684 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:43.685 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211967+0000", 2024-04-25T20:09:43.685 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8326999999999999e-05 2024-04-25T20:09:43.685 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.685 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.685 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:43.685 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211975+0000", 2024-04-25T20:09:43.685 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8589999999999994e-06 2024-04-25T20:09:43.685 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.685 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.685 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:43.685 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212020+0000", 2024-04-25T20:09:43.685 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4626999999999998e-05 2024-04-25T20:09:43.685 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.685 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.686 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:43.686 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212045+0000", 2024-04-25T20:09:43.686 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4508999999999999e-05 2024-04-25T20:09:43.686 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.686 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.686 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:43.686 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218316+0000", 2024-04-25T20:09:43.686 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0062711640000000001 2024-04-25T20:09:43.686 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.686 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.686 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:43.686 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218356+0000", 2024-04-25T20:09:43.686 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9993000000000001e-05 2024-04-25T20:09:43.687 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.687 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:43.687 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.687 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.687 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:68 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-04-25T20:09:43.687 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:09:43.687 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 26.436410442, 2024-04-25T20:09:43.687 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011231365, 2024-04-25T20:09:43.687 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:43.687 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:43.687 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:43.688 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.688 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:43.688 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:09:43.688 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.688 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.688 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:43.688 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:09:43.688 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.688 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.688 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:43.688 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:09:43.688 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:43.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.689 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.689 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:43.689 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212392+0000", 2024-04-25T20:09:43.689 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.79e-06 2024-04-25T20:09:43.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.689 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.689 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:43.689 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212406+0000", 2024-04-25T20:09:43.689 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3577000000000001e-05 2024-04-25T20:09:43.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.689 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.689 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:43.689 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212414+0000", 2024-04-25T20:09:43.689 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6510000000000001e-06 2024-04-25T20:09:43.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.690 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.690 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:43.690 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217114+0000", 2024-04-25T20:09:43.690 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047005379999999998 2024-04-25T20:09:43.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.690 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.690 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:43.690 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217141+0000", 2024-04-25T20:09:43.690 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6984000000000001e-05 2024-04-25T20:09:43.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.690 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.690 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:43.690 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223563+0000", 2024-04-25T20:09:43.690 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064216999999999998 2024-04-25T20:09:43.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.691 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.691 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:43.691 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223617+0000", 2024-04-25T20:09:43.691 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4011000000000002e-05 2024-04-25T20:09:43.691 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.691 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:43.691 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.691 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.691 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.691 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:69 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-04-25T20:09:43.691 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:09:43.691 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 26.435878251999998, 2024-04-25T20:09:43.691 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.010730094000000001, 2024-04-25T20:09:43.691 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:43.691 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:43.691 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:43.692 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.692 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:43.692 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:09:43.692 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.692 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.692 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:43.692 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:09:43.692 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.692 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.692 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:43.692 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212917+0000", 2024-04-25T20:09:43.692 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:43.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.692 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.693 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:43.693 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212957+0000", 2024-04-25T20:09:43.693 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0692000000000003e-05 2024-04-25T20:09:43.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.693 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.693 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:43.693 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212973+0000", 2024-04-25T20:09:43.693 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.575e-05 2024-04-25T20:09:43.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.693 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.693 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:43.693 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212980+0000", 2024-04-25T20:09:43.693 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1679999999999998e-06 2024-04-25T20:09:43.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.693 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.693 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:43.694 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217449+0000", 2024-04-25T20:09:43.694 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.004468367 2024-04-25T20:09:43.694 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.694 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.694 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:43.694 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217470+0000", 2024-04-25T20:09:43.694 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0749000000000001e-05 2024-04-25T20:09:43.694 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.694 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.694 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:43.694 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223621+0000", 2024-04-25T20:09:43.694 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0061512880000000004 2024-04-25T20:09:43.694 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.694 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.695 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:43.695 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223648+0000", 2024-04-25T20:09:43.695 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6961000000000002e-05 2024-04-25T20:09:43.695 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.695 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:43.695 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.695 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.695 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:70 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'44, mlcod=16'39)", 2024-04-25T20:09:43.695 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:09:43.695 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 26.435119524000001, 2024-04-25T20:09:43.695 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.013553358, 2024-04-25T20:09:43.695 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:43.695 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:43.696 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:43.696 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.696 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:43.696 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:09:43.696 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.696 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.696 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:43.696 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:09:43.696 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.696 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.696 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:43.697 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:09:43.697 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:43.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.697 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.697 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:43.697 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213682+0000", 2024-04-25T20:09:43.697 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3439999999999997e-06 2024-04-25T20:09:43.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.697 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.697 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:43.697 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213693+0000", 2024-04-25T20:09:43.697 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1049999999999999e-05 2024-04-25T20:09:43.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.698 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.698 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:43.698 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213699+0000", 2024-04-25T20:09:43.698 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9399999999999999e-06 2024-04-25T20:09:43.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.698 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.698 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:43.698 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218086+0000", 2024-04-25T20:09:43.698 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0043871240000000001 2024-04-25T20:09:43.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.698 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.698 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:43.698 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218106+0000", 2024-04-25T20:09:43.698 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.997e-05 2024-04-25T20:09:43.699 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.699 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.699 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:43.699 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.227186+0000", 2024-04-25T20:09:43.699 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.009079574 2024-04-25T20:09:43.699 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.699 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.699 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:43.699 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.227230+0000", 2024-04-25T20:09:43.699 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4147999999999998e-05 2024-04-25T20:09:43.699 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.699 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:43.699 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.700 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.700 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:106 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-04-25T20:09:43.700 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:09:43.700 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 26.063980424, 2024-04-25T20:09:43.700 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002936756, 2024-04-25T20:09:43.700 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:43.700 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:43.700 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:43.700 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.700 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:43.700 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:09:43.700 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.700 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.701 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:43.701 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:09:43.701 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.701 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.701 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:43.701 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584815+0000", 2024-04-25T20:09:43.701 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:43.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.701 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.701 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:43.701 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584860+0000", 2024-04-25T20:09:43.701 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5161999999999999e-05 2024-04-25T20:09:43.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.702 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.702 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:43.702 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584874+0000", 2024-04-25T20:09:43.702 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4464000000000001e-05 2024-04-25T20:09:43.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.702 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.702 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:43.702 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584881+0000", 2024-04-25T20:09:43.702 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.409e-06 2024-04-25T20:09:43.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.702 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.702 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:43.702 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584915+0000", 2024-04-25T20:09:43.702 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4007000000000002e-05 2024-04-25T20:09:43.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.703 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.703 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:43.703 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584933+0000", 2024-04-25T20:09:43.703 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7737999999999999e-05 2024-04-25T20:09:43.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.703 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.703 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:43.703 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587717+0000", 2024-04-25T20:09:43.703 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002784711 2024-04-25T20:09:43.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.703 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.703 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:43.703 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587752+0000", 2024-04-25T20:09:43.704 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4915e-05 2024-04-25T20:09:43.704 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.704 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:43.704 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.704 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.704 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.704 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:107 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'69, mlcod=16'64)", 2024-04-25T20:09:43.704 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:09:43.704 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 26.063905122000001, 2024-04-25T20:09:43.704 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028906270000000002, 2024-04-25T20:09:43.704 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:43.704 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:43.704 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:43.704 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.704 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:43.705 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:09:43.705 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.705 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.705 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:43.705 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:09:43.705 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.705 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.705 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:43.705 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584890+0000", 2024-04-25T20:09:43.705 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:43.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.705 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.706 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:43.706 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584895+0000", 2024-04-25T20:09:43.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2069999999999997e-06 2024-04-25T20:09:43.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.706 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:43.706 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584906+0000", 2024-04-25T20:09:43.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.117e-05 2024-04-25T20:09:43.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.706 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:43.706 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584910+0000", 2024-04-25T20:09:43.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4190000000000004e-06 2024-04-25T20:09:43.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:43.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585310+0000", 2024-04-25T20:09:43.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00040002700000000001 2024-04-25T20:09:43.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:43.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585329+0000", 2024-04-25T20:09:43.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8838000000000002e-05 2024-04-25T20:09:43.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:43.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587754+0000", 2024-04-25T20:09:43.708 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024252689999999999 2024-04-25T20:09:43.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.708 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:43.708 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587781+0000", 2024-04-25T20:09:43.708 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7206999999999999e-05 2024-04-25T20:09:43.708 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.708 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:43.708 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.708 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:108 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'70, mlcod=16'64)", 2024-04-25T20:09:43.708 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:09:43.708 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 26.063879970999999, 2024-04-25T20:09:43.709 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028924879999999999, 2024-04-25T20:09:43.709 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:43.709 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:43.709 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:43.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.709 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:43.709 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:09:43.709 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.709 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:43.709 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:09:43.709 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:43.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:09:43.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:43.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:43.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584919+0000", 2024-04-25T20:09:43.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.557e-06 2024-04-25T20:09:43.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:43.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584925+0000", 2024-04-25T20:09:43.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8619999999999999e-06 2024-04-25T20:09:43.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.711 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:43.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584931+0000", 2024-04-25T20:09:43.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2539999999999998e-06 2024-04-25T20:09:43.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.711 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:43.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585550+0000", 2024-04-25T20:09:43.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00061821800000000002 2024-04-25T20:09:43.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.711 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:43.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585567+0000", 2024-04-25T20:09:43.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7499e-05 2024-04-25T20:09:43.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.712 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.712 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:43.712 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587783+0000", 2024-04-25T20:09:43.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0022156200000000002 2024-04-25T20:09:43.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.712 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.712 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:43.712 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587808+0000", 2024-04-25T20:09:43.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5729e-05 2024-04-25T20:09:43.712 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.712 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:43.712 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.713 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:112 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'74, mlcod=16'66)", 2024-04-25T20:09:43.713 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:09:43.713 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 26.061722349, 2024-04-25T20:09:43.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00318724, 2024-04-25T20:09:43.713 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:43.713 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:43.713 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:43.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.713 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:43.713 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:09:43.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:43.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:09:43.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:43.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587073+0000", 2024-04-25T20:09:43.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:43.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:43.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587136+0000", 2024-04-25T20:09:43.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2928999999999993e-05 2024-04-25T20:09:43.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:43.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587147+0000", 2024-04-25T20:09:43.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1182999999999999e-05 2024-04-25T20:09:43.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:43.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587153+0000", 2024-04-25T20:09:43.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9059999999999999e-06 2024-04-25T20:09:43.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:43.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587306+0000", 2024-04-25T20:09:43.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000152848 2024-04-25T20:09:43.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:43.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587330+0000", 2024-04-25T20:09:43.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4304e-05 2024-04-25T20:09:43.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:43.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.590230+0000", 2024-04-25T20:09:43.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029004629999999998 2024-04-25T20:09:43.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:43.717 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.590261+0000", 2024-04-25T20:09:43.717 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0323000000000001e-05 2024-04-25T20:09:43.717 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.717 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:43.717 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.717 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.717 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.717 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:118 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'80, mlcod=16'78)", 2024-04-25T20:09:43.717 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:09:43.717 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 26.056935769999999, 2024-04-25T20:09:43.717 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029618959999999999, 2024-04-25T20:09:43.717 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:43.718 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:43.718 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:43.718 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.718 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:43.718 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:09:43.718 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.718 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.718 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:43.718 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:09:43.718 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.718 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:43.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591859+0000", 2024-04-25T20:09:43.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:43.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:43.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591918+0000", 2024-04-25T20:09:43.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8842999999999999e-05 2024-04-25T20:09:43.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:43.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591933+0000", 2024-04-25T20:09:43.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4600000000000001e-05 2024-04-25T20:09:43.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:43.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591939+0000", 2024-04-25T20:09:43.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4559999999999997e-06 2024-04-25T20:09:43.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:43.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591990+0000", 2024-04-25T20:09:43.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0318999999999997e-05 2024-04-25T20:09:43.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:43.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.592008+0000", 2024-04-25T20:09:43.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8819000000000001e-05 2024-04-25T20:09:43.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.721 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.721 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:43.721 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.594784+0000", 2024-04-25T20:09:43.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027750850000000001 2024-04-25T20:09:43.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.721 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.721 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:43.721 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.594822+0000", 2024-04-25T20:09:43.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8522000000000001e-05 2024-04-25T20:09:43.721 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.721 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:43.721 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.722 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.722 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.722 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:139 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'97, mlcod=17'87)", 2024-04-25T20:09:43.722 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:09:43.722 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 25.522899164999998, 2024-04-25T20:09:43.722 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048546290000000001, 2024-04-25T20:09:43.722 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:43.722 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:43.722 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:43.722 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.722 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:43.722 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:09:43.722 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:43.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:09:43.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:43.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125896+0000", 2024-04-25T20:09:43.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:43.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:43.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125908+0000", 2024-04-25T20:09:43.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.137e-05 2024-04-25T20:09:43.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:43.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125915+0000", 2024-04-25T20:09:43.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3939999999999998e-06 2024-04-25T20:09:43.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:43.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125922+0000", 2024-04-25T20:09:43.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1930000000000004e-06 2024-04-25T20:09:43.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:43.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125959+0000", 2024-04-25T20:09:43.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6701e-05 2024-04-25T20:09:43.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:43.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125972+0000", 2024-04-25T20:09:43.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2578000000000001e-05 2024-04-25T20:09:43.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:43.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130719+0000", 2024-04-25T20:09:43.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047475800000000004 2024-04-25T20:09:43.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.726 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:43.726 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130751+0000", 2024-04-25T20:09:43.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2283000000000002e-05 2024-04-25T20:09:43.726 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.726 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:43.726 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.726 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:140 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'98, mlcod=17'87)", 2024-04-25T20:09:43.726 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:09:43.726 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 25.522473814000001, 2024-04-25T20:09:43.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048310990000000002, 2024-04-25T20:09:43.726 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:43.727 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:43.727 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:43.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.727 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:43.727 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:09:43.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.727 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:43.727 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:09:43.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.727 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:43.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:09:43.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:43.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:43.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126427+0000", 2024-04-25T20:09:43.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000105572 2024-04-25T20:09:43.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:43.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126435+0000", 2024-04-25T20:09:43.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1920000000000005e-06 2024-04-25T20:09:43.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:43.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126439+0000", 2024-04-25T20:09:43.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6950000000000002e-06 2024-04-25T20:09:43.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:43.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126471+0000", 2024-04-25T20:09:43.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1699000000000003e-05 2024-04-25T20:09:43.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:43.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126484+0000", 2024-04-25T20:09:43.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.343e-05 2024-04-25T20:09:43.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:43.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131131+0000", 2024-04-25T20:09:43.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046471480000000003 2024-04-25T20:09:43.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.730 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.730 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:43.730 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131153+0000", 2024-04-25T20:09:43.730 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1753e-05 2024-04-25T20:09:43.730 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.730 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:43.730 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.730 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.730 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:141 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'99, mlcod=17'87)", 2024-04-25T20:09:43.730 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:09:43.730 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 25.521818870000001, 2024-04-25T20:09:43.731 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.004496847, 2024-04-25T20:09:43.731 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:43.731 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:43.731 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:43.731 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.731 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:43.731 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:09:43.731 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.731 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.731 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:43.731 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:09:43.731 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:43.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.732 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:43.732 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:09:43.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:43.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.732 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:43.732 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126980+0000", 2024-04-25T20:09:43.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.619e-06 2024-04-25T20:09:43.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.732 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:43.732 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126987+0000", 2024-04-25T20:09:43.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7220000000000002e-06 2024-04-25T20:09:43.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:43.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126992+0000", 2024-04-25T20:09:43.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8679999999999998e-06 2024-04-25T20:09:43.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:43.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130772+0000", 2024-04-25T20:09:43.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037803120000000001 2024-04-25T20:09:43.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:43.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130786+0000", 2024-04-25T20:09:43.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3592e-05 2024-04-25T20:09:43.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:43.734 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131451+0000", 2024-04-25T20:09:43.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000665206 2024-04-25T20:09:43.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:43.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:43.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:43.734 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131474+0000", 2024-04-25T20:09:43.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2957999999999999e-05 2024-04-25T20:09:43.734 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.734 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:43.734 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.734 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:43.735 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:43.735 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:43.736 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T20:09:43.737 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:43.735+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:43.837 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:43.836+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:43.842 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:43.842 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:43.842 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:09:43.842 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:43.851 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T20:09:43.938 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:43.938+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:43.956 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:43.956 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:43.956 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:09:43.957 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:09:43.957 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:43.965 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T20:09:44.039 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:44.038+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:44.111 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:44.111 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:09:44.111 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:09:44.111 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:09:44.111 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:44.120 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T20:09:44.164 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:44.164+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:44.180 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-25T20:09:44.228 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:44.229 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:44.229 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:09:44.229 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:44.238 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T20:09:44.241 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:44.240+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:44.342 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:44.342+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:44.347 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:44.348 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:44.348 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:09:44.348 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:09:44.348 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:44.357 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T20:09:44.443 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:44.443+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:44.503 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:44.503 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:09:44.503 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:09:44.503 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:09:44.503 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:44.512 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T20:09:44.553 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:44.553+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:44.582 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T20:09:44.624 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:44.624 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:44.624 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:09:44.624 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:44.633 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T20:09:44.644 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:44.644+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:44.704 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:09:44.704 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:09:44.721 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:09:44.721 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:09:44.736 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:44.736 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:44.736 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:09:44.736 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:09:44.736 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:44.737 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:09:44.737 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:09:44.744 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T20:09:44.745 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:44.745+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:44.753 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:09:44.753 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:09:44.770 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:09:44.770 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:09:44.788 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:09:44.788 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:09:44.804 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:09:44.805 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:09:44.821 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:09:44.821 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:09:44.847 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:44.846+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:44.848 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:44.848 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:44.857 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T20:09:44.947 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:44.947+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:44.998 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:44.998 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:45.006 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T20:09:45.048 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:45.047+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:45.148 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:45.148 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:45.149 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:45.148+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:45.157 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T20:09:45.180 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is unset 2024-04-25T20:09:45.186 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6, 7] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-25T20:09:45.186 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T20:09:45.187 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-04-25T20:09:45.187 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-04-25T20:09:45.187 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-04-25T20:09:45.187 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-04-25T20:09:45.249 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:45.249+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:45.334 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:45.334 INFO:teuthology.orchestra.run.smithi135.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-04-25T20:09:45.334 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:45.336 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:45.336 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:45.336 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T20:09:45.351 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:45.350+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:45.452 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:45.452+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:45.478 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:45.478 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:45.487 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T20:09:45.553 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:45.552+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:45.588 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:45.588 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:45.596 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T20:09:45.653 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:45.653+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:45.699 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:45.699 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:45.708 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T20:09:45.754 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:45.754+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:45.809 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:45.809 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:45.818 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T20:09:45.855 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:45.854+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:45.919 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:45.919 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:45.928 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T20:09:45.955 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:45.955+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:46.029 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:46.029 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:46.038 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T20:09:46.056 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:46.055+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:46.156 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:46.156 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:46.157 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:46.156+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:46.165 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T20:09:46.258 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:46.256+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:46.313 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:46.313 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:46.322 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T20:09:46.358 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:46.357+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:46.427 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:46.427 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:46.436 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T20:09:46.479 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:46.479+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:46.542 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:46.542 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:46.551 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T20:09:46.592 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:46.592+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:46.660 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:46.660+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:46.696 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:46.697 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:46.706 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T20:09:46.790 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:46.790+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:46.833 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T20:09:46.853 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:46.854 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:46.861 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:46.861+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:46.864 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T20:09:46.956 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:09:46.957 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:09:46.962 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:46.961+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:46.974 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:09:46.974 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:09:46.990 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:09:46.990 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:09:47.007 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:09:47.007 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:09:47.013 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:47.013 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:47.013 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:09:47.013 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:47.022 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T20:09:47.024 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:09:47.024 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:09:47.042 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:09:47.042 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:09:47.059 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:09:47.059 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:09:47.062 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:47.062+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:47.077 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:09:47.077 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:09:47.129 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:47.129 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:47.129 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:09:47.129 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:09:47.129 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:47.138 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T20:09:47.163 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:47.163+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:47.194 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-25T20:09:47.245 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:47.245 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:09:47.245 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:09:47.245 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:09:47.245 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:47.255 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T20:09:47.264 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:47.264+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:47.365 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:47.364+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:47.366 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:47.366 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:47.366 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:09:47.367 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:47.375 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T20:09:47.466 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:47.465+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:47.522 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:47.522 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:47.522 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:09:47.523 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:09:47.523 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:47.531 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T20:09:47.567 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:47.566+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:47.637 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:47.637 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:09:47.637 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:09:47.637 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:09:47.637 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:47.645 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T20:09:47.667 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:47.667+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:47.759 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:47.759 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:47.759 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:09:47.759 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:47.768 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T20:09:47.769 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:47.768+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:47.869 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:47.869+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:47.871 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:47.871 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:47.871 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:09:47.872 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:09:47.872 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:47.880 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T20:09:47.971 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:47.970+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:47.981 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:47.981 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:09:47.982 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:09:47.982 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:09:47.982 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:47.990 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T20:09:48.071 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:48.070+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:48.094 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:48.094 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:48.095 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:09:48.095 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:48.104 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T20:09:48.189 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:48.188+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:48.201 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is unset 2024-04-25T20:09:48.247 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:48.247 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:48.247 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:09:48.247 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:09:48.247 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:48.256 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T20:09:48.273 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:48.273+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:48.373 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:48.372+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:48.399 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:48.399 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:09:48.399 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:09:48.400 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:09:48.400 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:48.408 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T20:09:48.492 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:48.491+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:48.550 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:48.550 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:48.550 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:09:48.550 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:48.559 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T20:09:48.575 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:48.575+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:48.676 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:48.675+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:48.700 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:48.700 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:48.700 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:09:48.701 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:09:48.701 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:48.709 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T20:09:48.794 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:48.793+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:48.852 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:48.852 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:09:48.852 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:09:48.852 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:09:48.852 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:48.861 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T20:09:48.877 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:48.877+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:48.978 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:48.977+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:49.004 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:49.004 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:49.005 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:09:49.005 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:49.014 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T20:09:49.089 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T20:09:49.099 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:49.098+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:49.155 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:49.155 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:49.155 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:09:49.156 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:09:49.156 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:49.164 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T20:09:49.205 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:49.204+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:49.227 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:09:49.227 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:09:49.244 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:09:49.244 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:09:49.260 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:09:49.261 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:09:49.277 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:09:49.277 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:09:49.281 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:49.280+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:49.293 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:09:49.294 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:09:49.308 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:49.308 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:09:49.308 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:09:49.308 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [ 2024-04-25T20:09:49.308 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.308 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:09:49.308 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:09:49.308 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 32.524097320999999, 2024-04-25T20:09:49.308 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.34154860100000001, 2024-04-25T20:09:49.308 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:49.308 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:49.308 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:49.309 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:49.309 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:49.309 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 1 2024-04-25T20:09:49.309 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.309 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:49.309 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.309 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:49.309 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:09:49.309 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.309 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.309 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.309 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:49.309 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:09:49.309 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.310 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.310 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.310 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:49.310 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782261+0000", 2024-04-25T20:09:49.310 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999986 2024-04-25T20:09:49.310 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.310 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.310 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:49.310 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782268+0000", 2024-04-25T20:09:49.310 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.8140000000000001e-06 2024-04-25T20:09:49.310 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.310 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.310 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:49.310 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782279+0000", 2024-04-25T20:09:49.310 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0665999999999999e-05 2024-04-25T20:09:49.311 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.311 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.311 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:49.311 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782290+0000", 2024-04-25T20:09:49.311 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.1343e-05 2024-04-25T20:09:49.311 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.311 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.311 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:49.311 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782340+0000", 2024-04-25T20:09:49.311 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.9311999999999998e-05 2024-04-25T20:09:49.311 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.311 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.311 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for peered", 2024-04-25T20:09:49.312 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782348+0000", 2024-04-25T20:09:49.312 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.0690000000000002e-06 2024-04-25T20:09:49.312 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.312 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.312 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:49.312 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120520+0000", 2024-04-25T20:09:49.312 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.33817188100000001 2024-04-25T20:09:49.312 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.312 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.312 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:49.312 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120656+0000", 2024-04-25T20:09:49.312 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000135897 2024-04-25T20:09:49.312 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.312 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.312 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:49.313 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120802+0000", 2024-04-25T20:09:49.313 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00014691199999999999 2024-04-25T20:09:49.313 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.313 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.313 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:49.313 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.122368+0000", 2024-04-25T20:09:49.313 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0015651040000000001 2024-04-25T20:09:49.313 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.313 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.313 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:49.313 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123729+0000", 2024-04-25T20:09:49.313 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0013610499999999999 2024-04-25T20:09:49.313 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.313 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.313 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:49.313 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123769+0000", 2024-04-25T20:09:49.314 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0719000000000001e-05 2024-04-25T20:09:49.314 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.314 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.314 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:49.314 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123811+0000", 2024-04-25T20:09:49.314 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1267999999999999e-05 2024-04-25T20:09:49.314 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.314 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:49.314 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.314 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.314 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.314 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:67 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:09:49.314 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:09:49.314 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 32.094678434000002, 2024-04-25T20:09:49.314 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.006947651, 2024-04-25T20:09:49.315 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:49.315 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:49.315 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:49.315 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:49.315 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:49.315 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 67 2024-04-25T20:09:49.315 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.315 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:49.315 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.315 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:49.315 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:09:49.315 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.315 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.315 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.315 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:49.316 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:09:49.316 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.316 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.316 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.316 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:49.316 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211680+0000", 2024-04-25T20:09:49.316 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:49.316 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.316 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.316 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:49.316 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211684+0000", 2024-04-25T20:09:49.316 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4249999999999998e-06 2024-04-25T20:09:49.316 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.316 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.316 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:49.317 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211690+0000", 2024-04-25T20:09:49.317 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.7360000000000004e-06 2024-04-25T20:09:49.317 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.317 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.317 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:49.317 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211694+0000", 2024-04-25T20:09:49.317 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4860000000000002e-06 2024-04-25T20:09:49.317 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.317 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.317 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:49.317 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211727+0000", 2024-04-25T20:09:49.317 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3445999999999998e-05 2024-04-25T20:09:49.317 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.317 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.317 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:49.317 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211778+0000", 2024-04-25T20:09:49.318 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1050000000000001e-05 2024-04-25T20:09:49.318 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.318 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.318 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:49.318 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211851+0000", 2024-04-25T20:09:49.318 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2914e-05 2024-04-25T20:09:49.318 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.318 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.318 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:49.318 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213824+0000", 2024-04-25T20:09:49.318 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0019726520000000001 2024-04-25T20:09:49.318 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.318 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.318 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:49.319 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218579+0000", 2024-04-25T20:09:49.319 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0047553159999999999 2024-04-25T20:09:49.319 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.319 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.319 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:49.319 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218607+0000", 2024-04-25T20:09:49.319 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8144999999999999e-05 2024-04-25T20:09:49.319 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.319 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.319 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:49.319 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218629+0000", 2024-04-25T20:09:49.319 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.1044e-05 2024-04-25T20:09:49.319 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.319 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:49.319 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.320 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.320 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.320 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:68 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:09:49.320 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:09:49.320 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 32.094660093000002, 2024-04-25T20:09:49.320 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012255596000000001, 2024-04-25T20:09:49.320 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:49.320 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:49.320 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:49.320 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:49.320 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:49.320 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 68 2024-04-25T20:09:49.320 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.320 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:49.320 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.321 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:49.321 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:09:49.321 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.321 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.321 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.321 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:49.321 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:09:49.321 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.321 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.321 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.321 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:49.321 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:09:49.321 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:49.321 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.321 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.322 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:49.322 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211701+0000", 2024-04-25T20:09:49.322 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2029999999999999e-06 2024-04-25T20:09:49.322 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.322 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.322 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:49.322 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211705+0000", 2024-04-25T20:09:49.322 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.2320000000000003e-06 2024-04-25T20:09:49.322 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.322 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.322 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:49.322 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211708+0000", 2024-04-25T20:09:49.322 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.6290000000000002e-06 2024-04-25T20:09:49.322 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.323 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.323 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:49.323 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212136+0000", 2024-04-25T20:09:49.323 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00042764099999999998 2024-04-25T20:09:49.323 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.323 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.323 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:49.323 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212212+0000", 2024-04-25T20:09:49.323 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.6315000000000004e-05 2024-04-25T20:09:49.323 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.323 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.323 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:49.323 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212306+0000", 2024-04-25T20:09:49.323 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.3556999999999999e-05 2024-04-25T20:09:49.323 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.324 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.324 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:49.324 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213829+0000", 2024-04-25T20:09:49.324 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001523256 2024-04-25T20:09:49.324 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.324 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.324 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:49.324 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223904+0000", 2024-04-25T20:09:49.324 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010075387 2024-04-25T20:09:49.324 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.324 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.324 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:49.324 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223934+0000", 2024-04-25T20:09:49.324 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9988000000000002e-05 2024-04-25T20:09:49.324 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.325 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.325 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:49.325 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223955+0000", 2024-04-25T20:09:49.325 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0630999999999999e-05 2024-04-25T20:09:49.325 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.325 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:49.325 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.325 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.325 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.325 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:69 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:09:49.325 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:09:49.325 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 32.094638357999997, 2024-04-25T20:09:49.325 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012316863000000001, 2024-04-25T20:09:49.325 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:49.326 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:49.326 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:49.326 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:49.326 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:49.326 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 69 2024-04-25T20:09:49.326 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.326 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:49.326 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.326 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:49.326 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:09:49.326 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.326 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.326 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.327 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:49.327 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:09:49.327 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.327 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.327 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.327 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:49.327 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:09:49.327 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:49.327 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.327 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.327 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:49.327 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211762+0000", 2024-04-25T20:09:49.327 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1703999999999999e-05 2024-04-25T20:09:49.327 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.328 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.328 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:49.328 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211768+0000", 2024-04-25T20:09:49.328 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.3430000000000003e-06 2024-04-25T20:09:49.328 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.328 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.328 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:49.328 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211771+0000", 2024-04-25T20:09:49.328 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.18e-06 2024-04-25T20:09:49.328 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.328 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.328 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:49.328 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212652+0000", 2024-04-25T20:09:49.328 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00088095800000000004 2024-04-25T20:09:49.328 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.328 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.328 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:49.329 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212703+0000", 2024-04-25T20:09:49.329 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.0655999999999997e-05 2024-04-25T20:09:49.329 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.329 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.329 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:49.329 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212781+0000", 2024-04-25T20:09:49.329 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.8852999999999999e-05 2024-04-25T20:09:49.329 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.329 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.329 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:49.329 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.214131+0000", 2024-04-25T20:09:49.329 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0013496039999999999 2024-04-25T20:09:49.329 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.329 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.329 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:49.329 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223986+0000", 2024-04-25T20:09:49.330 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.009854804 2024-04-25T20:09:49.330 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.330 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.330 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:49.330 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.224017+0000", 2024-04-25T20:09:49.330 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.1068999999999997e-05 2024-04-25T20:09:49.330 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.330 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.330 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:49.330 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.224038+0000", 2024-04-25T20:09:49.330 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0965e-05 2024-04-25T20:09:49.330 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.330 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:49.330 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.331 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.331 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.331 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:70 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:09:49.331 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:09:49.331 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 32.094583276999998, 2024-04-25T20:09:49.331 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.015776348999999999, 2024-04-25T20:09:49.331 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:49.331 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:49.331 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:49.331 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:49.331 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:49.331 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 70 2024-04-25T20:09:49.331 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.331 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:49.332 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.332 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:49.332 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:09:49.332 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.332 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.332 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.332 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:49.332 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:09:49.332 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.332 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.332 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.332 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:49.332 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:09:49.332 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:49.332 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.333 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.333 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:49.333 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211778+0000", 2024-04-25T20:09:49.333 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5330000000000001e-06 2024-04-25T20:09:49.333 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.333 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.333 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:49.333 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211782+0000", 2024-04-25T20:09:49.333 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4079999999999998e-06 2024-04-25T20:09:49.333 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.333 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.333 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:49.333 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211786+0000", 2024-04-25T20:09:49.333 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.6190000000000002e-06 2024-04-25T20:09:49.333 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.333 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.334 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:49.334 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213374+0000", 2024-04-25T20:09:49.334 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0015881859999999999 2024-04-25T20:09:49.334 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.334 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.334 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:49.334 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213478+0000", 2024-04-25T20:09:49.334 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000103464 2024-04-25T20:09:49.334 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.334 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.334 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:49.334 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213601+0000", 2024-04-25T20:09:49.334 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00012324300000000001 2024-04-25T20:09:49.334 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.335 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.335 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:49.335 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.214788+0000", 2024-04-25T20:09:49.335 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011868569999999999 2024-04-25T20:09:49.335 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.335 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.335 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:49.335 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227504+0000", 2024-04-25T20:09:49.335 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012715749 2024-04-25T20:09:49.335 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.335 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.335 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:49.335 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227532+0000", 2024-04-25T20:09:49.335 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.7937e-05 2024-04-25T20:09:49.335 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.335 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.336 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:49.336 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227552+0000", 2024-04-25T20:09:49.336 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0772999999999999e-05 2024-04-25T20:09:49.336 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.336 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:49.336 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.336 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.336 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.336 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:131 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:49.336 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:09:49.336 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 31.185573432000002, 2024-04-25T20:09:49.336 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0086158810000000006, 2024-04-25T20:09:49.336 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:49.336 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:49.336 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:49.336 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:49.336 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:49.337 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 131 2024-04-25T20:09:49.337 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.337 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:49.337 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.337 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:49.337 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:09:49.337 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.337 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.337 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.337 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:49.337 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:09:49.337 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.337 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.337 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.338 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:49.338 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120785+0000", 2024-04-25T20:09:49.338 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:49.338 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.338 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.338 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:49.338 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120880+0000", 2024-04-25T20:09:49.338 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.4925000000000003e-05 2024-04-25T20:09:49.338 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.338 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.338 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:49.338 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120891+0000", 2024-04-25T20:09:49.338 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0915e-05 2024-04-25T20:09:49.338 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.339 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.339 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:49.339 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120895+0000", 2024-04-25T20:09:49.339 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.8070000000000001e-06 2024-04-25T20:09:49.339 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.339 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.339 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:49.339 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121640+0000", 2024-04-25T20:09:49.339 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00074468500000000003 2024-04-25T20:09:49.339 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.339 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.339 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:49.339 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121684+0000", 2024-04-25T20:09:49.339 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4452000000000001e-05 2024-04-25T20:09:49.339 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.339 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.339 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:49.340 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121753+0000", 2024-04-25T20:09:49.340 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8508000000000002e-05 2024-04-25T20:09:49.340 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.340 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.340 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:49.340 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123618+0000", 2024-04-25T20:09:49.340 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0018655099999999999 2024-04-25T20:09:49.340 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.340 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.340 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:49.340 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129248+0000", 2024-04-25T20:09:49.340 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0056295540000000002 2024-04-25T20:09:49.340 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.340 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.340 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:49.340 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129368+0000", 2024-04-25T20:09:49.341 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000120244 2024-04-25T20:09:49.341 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.341 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.341 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:49.341 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129402+0000", 2024-04-25T20:09:49.341 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3812000000000001e-05 2024-04-25T20:09:49.341 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.341 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:49.341 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.341 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.341 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.341 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:132 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:49.341 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:09:49.341 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 31.185459221999999, 2024-04-25T20:09:49.341 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0085898169999999996, 2024-04-25T20:09:49.341 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:49.341 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:49.342 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:49.342 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:49.342 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:49.342 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 132 2024-04-25T20:09:49.342 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.342 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:49.342 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.342 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:49.342 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:09:49.342 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.342 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.342 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.342 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:49.342 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:09:49.342 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.343 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.343 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.343 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:49.343 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:09:49.343 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:49.343 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.343 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.343 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:49.343 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120902+0000", 2024-04-25T20:09:49.343 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.6129999999999998e-06 2024-04-25T20:09:49.343 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.343 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.343 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:49.343 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120906+0000", 2024-04-25T20:09:49.343 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6569999999999999e-06 2024-04-25T20:09:49.343 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.344 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.344 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:49.344 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120909+0000", 2024-04-25T20:09:49.344 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5550000000000001e-06 2024-04-25T20:09:49.344 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.344 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.344 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:49.344 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122098+0000", 2024-04-25T20:09:49.344 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011898049999999999 2024-04-25T20:09:49.344 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.344 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.344 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:49.344 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122172+0000", 2024-04-25T20:09:49.344 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3917000000000004e-05 2024-04-25T20:09:49.344 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.345 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.345 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:49.345 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122288+0000", 2024-04-25T20:09:49.345 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00011566 2024-04-25T20:09:49.345 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.345 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.345 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:49.345 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124519+0000", 2024-04-25T20:09:49.345 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0022308359999999999 2024-04-25T20:09:49.345 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.345 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.345 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:49.345 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129414+0000", 2024-04-25T20:09:49.345 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048953720000000003 2024-04-25T20:09:49.345 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.346 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.346 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:49.346 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129458+0000", 2024-04-25T20:09:49.346 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.3426000000000001e-05 2024-04-25T20:09:49.346 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.346 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.346 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:49.346 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129490+0000", 2024-04-25T20:09:49.346 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2329999999999997e-05 2024-04-25T20:09:49.346 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.346 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:49.346 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.346 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.346 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.347 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:133 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:49.347 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:09:49.347 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 31.185405353, 2024-04-25T20:09:49.347 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0086066270000000004, 2024-04-25T20:09:49.347 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:49.347 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:49.347 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:49.347 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:49.347 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:49.347 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 133 2024-04-25T20:09:49.347 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.347 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:49.347 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.347 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:49.348 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:09:49.348 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.348 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.348 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.348 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:49.348 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:09:49.348 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.348 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.348 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.348 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:49.348 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:09:49.348 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:49.348 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.348 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.348 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:49.349 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120956+0000", 2024-04-25T20:09:49.349 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5289999999999998e-06 2024-04-25T20:09:49.349 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.349 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.349 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:49.349 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120962+0000", 2024-04-25T20:09:49.349 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.9490000000000001e-06 2024-04-25T20:09:49.349 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.349 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.349 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:49.349 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120965+0000", 2024-04-25T20:09:49.349 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.1860000000000001e-06 2024-04-25T20:09:49.349 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.349 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.349 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:49.350 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122666+0000", 2024-04-25T20:09:49.350 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0017004500000000001 2024-04-25T20:09:49.350 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.350 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.350 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:49.350 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122741+0000", 2024-04-25T20:09:49.350 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.5297999999999996e-05 2024-04-25T20:09:49.350 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.350 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.350 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:49.350 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122858+0000", 2024-04-25T20:09:49.350 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00011723899999999999 2024-04-25T20:09:49.350 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.350 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.350 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:49.351 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124625+0000", 2024-04-25T20:09:49.351 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0017670100000000001 2024-04-25T20:09:49.351 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.351 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.351 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:49.351 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129497+0000", 2024-04-25T20:09:49.351 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048713580000000001 2024-04-25T20:09:49.351 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.351 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.351 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:49.351 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129532+0000", 2024-04-25T20:09:49.351 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5704999999999998e-05 2024-04-25T20:09:49.351 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.351 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.352 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:49.352 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129561+0000", 2024-04-25T20:09:49.352 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8283e-05 2024-04-25T20:09:49.352 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.352 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:49.352 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.352 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.352 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.352 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:134 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:49.352 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:09:49.352 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 31.185303155, 2024-04-25T20:09:49.352 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0085771179999999999, 2024-04-25T20:09:49.352 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:49.352 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:49.353 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:49.353 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:49.353 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:49.353 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 134 2024-04-25T20:09:49.353 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.353 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:49.353 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.353 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:49.353 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:09:49.353 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.353 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.353 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.353 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:49.353 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:09:49.353 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.354 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.354 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.354 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:49.354 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:09:49.354 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:49.354 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.354 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.354 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:49.354 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121157+0000", 2024-04-25T20:09:49.354 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00010104599999999999 2024-04-25T20:09:49.354 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.354 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.354 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:49.354 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121189+0000", 2024-04-25T20:09:49.354 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2589000000000003e-05 2024-04-25T20:09:49.355 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.355 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.355 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:49.355 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121195+0000", 2024-04-25T20:09:49.355 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.083e-06 2024-04-25T20:09:49.355 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.355 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.355 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:49.355 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123136+0000", 2024-04-25T20:09:49.355 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0019409780000000001 2024-04-25T20:09:49.355 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.355 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.355 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:49.355 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123181+0000", 2024-04-25T20:09:49.355 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4267999999999997e-05 2024-04-25T20:09:49.356 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.356 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.356 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:49.356 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123247+0000", 2024-04-25T20:09:49.356 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.6086000000000004e-05 2024-04-25T20:09:49.356 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.356 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.356 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:49.356 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125639+0000", 2024-04-25T20:09:49.356 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.002392 2024-04-25T20:09:49.356 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.356 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.356 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:49.356 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129567+0000", 2024-04-25T20:09:49.356 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0039282170000000003 2024-04-25T20:09:49.356 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.357 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.357 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:49.357 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129605+0000", 2024-04-25T20:09:49.357 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.7660999999999997e-05 2024-04-25T20:09:49.357 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.357 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.357 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:49.357 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129633+0000", 2024-04-25T20:09:49.357 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8649999999999998e-05 2024-04-25T20:09:49.357 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.357 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:49.357 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.357 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.357 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.357 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:135 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:49.358 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:09:49.358 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 31.185144148999999, 2024-04-25T20:09:49.358 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010898453000000001, 2024-04-25T20:09:49.358 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:49.358 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:49.358 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:49.358 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:49.358 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:49.358 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 135 2024-04-25T20:09:49.358 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.358 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:49.358 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.358 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:49.358 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:09:49.358 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.359 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.359 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.359 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:49.359 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:09:49.359 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.359 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.359 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.359 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:49.359 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121214+0000", 2024-04-25T20:09:49.359 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:49.359 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.359 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.359 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:49.359 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121218+0000", 2024-04-25T20:09:49.359 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6619999999999998e-06 2024-04-25T20:09:49.360 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.360 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.360 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:49.360 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121223+0000", 2024-04-25T20:09:49.360 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1070000000000004e-06 2024-04-25T20:09:49.360 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.360 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.360 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:49.360 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121227+0000", 2024-04-25T20:09:49.360 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2820000000000001e-06 2024-04-25T20:09:49.360 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.360 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.360 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:49.360 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123649+0000", 2024-04-25T20:09:49.360 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0024221970000000001 2024-04-25T20:09:49.360 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.361 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.361 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:49.361 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123723+0000", 2024-04-25T20:09:49.361 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.4386000000000002e-05 2024-04-25T20:09:49.361 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.361 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.361 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:49.361 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123844+0000", 2024-04-25T20:09:49.361 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000121249 2024-04-25T20:09:49.361 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.361 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.361 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:49.361 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125670+0000", 2024-04-25T20:09:49.361 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001825901 2024-04-25T20:09:49.362 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.362 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.362 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:49.362 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132043+0000", 2024-04-25T20:09:49.362 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0063724539999999996 2024-04-25T20:09:49.362 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.362 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.362 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:49.362 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132084+0000", 2024-04-25T20:09:49.362 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0963000000000001e-05 2024-04-25T20:09:49.362 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.362 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.362 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:49.362 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132114+0000", 2024-04-25T20:09:49.363 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9927e-05 2024-04-25T20:09:49.363 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.363 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:49.363 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.363 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.363 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.363 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:136 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:49.363 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:09:49.363 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 31.185128623000001, 2024-04-25T20:09:49.363 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010952513000000001, 2024-04-25T20:09:49.363 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:49.363 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:49.363 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:49.363 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:49.364 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:49.364 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 136 2024-04-25T20:09:49.364 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.364 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:49.364 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.364 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:49.364 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:09:49.364 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.364 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.364 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.364 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:49.364 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:09:49.364 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.364 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.364 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.365 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:49.365 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121230+0000", 2024-04-25T20:09:49.365 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:49.365 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.365 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.365 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:49.365 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121236+0000", 2024-04-25T20:09:49.365 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.7320000000000003e-06 2024-04-25T20:09:49.365 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.365 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.365 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:49.365 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121242+0000", 2024-04-25T20:09:49.365 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.7810000000000002e-06 2024-04-25T20:09:49.365 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.365 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.365 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:49.366 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121246+0000", 2024-04-25T20:09:49.366 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.639e-06 2024-04-25T20:09:49.366 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.366 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.366 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:49.366 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124150+0000", 2024-04-25T20:09:49.366 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.002903816 2024-04-25T20:09:49.366 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.366 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.366 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:49.366 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124195+0000", 2024-04-25T20:09:49.366 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.5042e-05 2024-04-25T20:09:49.366 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.366 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.367 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:49.367 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124266+0000", 2024-04-25T20:09:49.367 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0234999999999994e-05 2024-04-25T20:09:49.367 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.367 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.367 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:49.367 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125711+0000", 2024-04-25T20:09:49.367 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0014458800000000001 2024-04-25T20:09:49.367 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.367 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.367 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:49.367 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132120+0000", 2024-04-25T20:09:49.367 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0064084399999999996 2024-04-25T20:09:49.367 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.367 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.367 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:49.367 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132155+0000", 2024-04-25T20:09:49.368 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4938000000000003e-05 2024-04-25T20:09:49.368 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.368 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.368 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:49.368 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132183+0000", 2024-04-25T20:09:49.368 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8360999999999998e-05 2024-04-25T20:09:49.368 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.368 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:49.368 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.368 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.368 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.368 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:49.368 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:09:49.369 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 31.185078891, 2024-04-25T20:09:49.369 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.01097684, 2024-04-25T20:09:49.369 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:49.369 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:49.369 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:49.369 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:49.369 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:49.369 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 137 2024-04-25T20:09:49.369 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.369 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:49.369 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.369 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:49.369 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:09:49.369 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.369 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.370 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.370 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:49.370 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:09:49.370 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.370 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.370 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.370 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:49.370 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:09:49.370 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:49.370 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.370 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.370 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:49.370 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121328+0000", 2024-04-25T20:09:49.370 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.7908999999999998e-05 2024-04-25T20:09:49.370 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.371 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.371 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:49.371 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121337+0000", 2024-04-25T20:09:49.371 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.9369999999999993e-06 2024-04-25T20:09:49.371 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.371 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.371 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:49.371 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121343+0000", 2024-04-25T20:09:49.371 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.8089999999999998e-06 2024-04-25T20:09:49.371 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.371 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.371 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:49.371 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124656+0000", 2024-04-25T20:09:49.371 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0033133839999999999 2024-04-25T20:09:49.371 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.371 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.372 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:49.372 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124730+0000", 2024-04-25T20:09:49.372 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3565000000000005e-05 2024-04-25T20:09:49.372 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.372 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.372 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:49.372 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124846+0000", 2024-04-25T20:09:49.372 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000116416 2024-04-25T20:09:49.372 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.372 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.372 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:49.372 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126756+0000", 2024-04-25T20:09:49.372 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001909991 2024-04-25T20:09:49.372 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.372 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.373 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:49.373 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132190+0000", 2024-04-25T20:09:49.373 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0054343880000000001 2024-04-25T20:09:49.373 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.373 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.373 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:49.373 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132227+0000", 2024-04-25T20:09:49.373 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6875000000000002e-05 2024-04-25T20:09:49.373 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.373 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.373 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:49.373 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132257+0000", 2024-04-25T20:09:49.373 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.0046e-05 2024-04-25T20:09:49.373 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.373 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:49.374 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.374 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.374 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.374 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:49.374 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:09:49.374 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 31.185008932999999, 2024-04-25T20:09:49.374 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010978653999999999, 2024-04-25T20:09:49.374 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:49.374 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:49.374 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:49.374 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:49.374 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:49.374 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 138 2024-04-25T20:09:49.374 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.375 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:49.375 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.375 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:49.375 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:09:49.375 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.375 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.375 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.375 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:49.375 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:09:49.375 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.375 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.375 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.375 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:49.375 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:09:49.375 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:49.375 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.375 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.376 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:49.376 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121355+0000", 2024-04-25T20:09:49.376 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.9869999999999999e-06 2024-04-25T20:09:49.376 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.376 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.376 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:49.376 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121361+0000", 2024-04-25T20:09:49.376 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.4620000000000001e-06 2024-04-25T20:09:49.376 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.376 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.376 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:49.376 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121373+0000", 2024-04-25T20:09:49.376 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.2035000000000001e-05 2024-04-25T20:09:49.376 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.376 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.376 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:49.377 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125267+0000", 2024-04-25T20:09:49.377 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0038939230000000001 2024-04-25T20:09:49.377 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.377 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.377 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:49.377 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125309+0000", 2024-04-25T20:09:49.377 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1690999999999998e-05 2024-04-25T20:09:49.377 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.377 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.377 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:49.377 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125385+0000", 2024-04-25T20:09:49.377 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.6162999999999996e-05 2024-04-25T20:09:49.377 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.377 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.377 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:49.377 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126798+0000", 2024-04-25T20:09:49.377 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0014124680000000001 2024-04-25T20:09:49.378 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.378 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.378 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:49.378 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132264+0000", 2024-04-25T20:09:49.378 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0054661689999999999 2024-04-25T20:09:49.378 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.378 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.378 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:49.378 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132299+0000", 2024-04-25T20:09:49.378 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5272999999999999e-05 2024-04-25T20:09:49.378 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.378 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.378 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:49.378 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132329+0000", 2024-04-25T20:09:49.379 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.0080999999999999e-05 2024-04-25T20:09:49.379 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.379 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:49.379 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.379 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.379 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.379 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:49.379 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:09:49.379 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 31.184978357999999, 2024-04-25T20:09:49.379 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011015853000000001, 2024-04-25T20:09:49.379 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:49.379 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:49.379 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:49.379 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:49.379 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:49.379 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 139 2024-04-25T20:09:49.380 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.380 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:49.380 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.380 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:49.380 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:09:49.380 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.380 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.380 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.380 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:49.380 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:09:49.380 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.380 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.380 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.380 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:49.380 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121380+0000", 2024-04-25T20:09:49.380 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:49.381 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.381 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.381 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:49.381 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121385+0000", 2024-04-25T20:09:49.381 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.092e-06 2024-04-25T20:09:49.381 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.381 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.381 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:49.381 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121390+0000", 2024-04-25T20:09:49.381 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.5990000000000001e-06 2024-04-25T20:09:49.381 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.381 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.381 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:49.381 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121397+0000", 2024-04-25T20:09:49.381 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.6279999999999999e-06 2024-04-25T20:09:49.382 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.382 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.382 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:49.382 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125737+0000", 2024-04-25T20:09:49.382 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0043406010000000004 2024-04-25T20:09:49.382 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.382 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.382 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:49.382 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125781+0000", 2024-04-25T20:09:49.382 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.3958999999999997e-05 2024-04-25T20:09:49.382 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.382 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.382 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:49.382 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125849+0000", 2024-04-25T20:09:49.382 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8071999999999995e-05 2024-04-25T20:09:49.383 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.383 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.383 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:49.383 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.130908+0000", 2024-04-25T20:09:49.383 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0050586509999999999 2024-04-25T20:09:49.383 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.383 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.383 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:49.383 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132334+0000", 2024-04-25T20:09:49.383 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00142621 2024-04-25T20:09:49.383 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.383 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.383 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:49.383 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132369+0000", 2024-04-25T20:09:49.384 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4452000000000002e-05 2024-04-25T20:09:49.384 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.384 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.384 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:49.384 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132397+0000", 2024-04-25T20:09:49.384 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8065e-05 2024-04-25T20:09:49.384 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.384 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:49.384 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.384 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.384 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.384 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:49.384 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:09:49.384 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 31.184957313999998, 2024-04-25T20:09:49.384 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011069443, 2024-04-25T20:09:49.385 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:49.385 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:49.385 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:49.385 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:49.385 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:49.385 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 140 2024-04-25T20:09:49.385 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.385 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:49.385 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.385 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:49.385 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:09:49.385 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.385 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.385 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.385 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:49.385 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:09:49.385 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.386 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.386 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.386 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:49.386 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:09:49.386 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:49.386 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.386 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.386 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:49.386 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121456+0000", 2024-04-25T20:09:49.386 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.4469000000000003e-05 2024-04-25T20:09:49.386 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.386 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.386 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:49.386 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121462+0000", 2024-04-25T20:09:49.386 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.1890000000000004e-06 2024-04-25T20:09:49.387 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.387 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.387 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:49.387 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121466+0000", 2024-04-25T20:09:49.387 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3909999999999998e-06 2024-04-25T20:09:49.387 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.387 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.387 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:49.387 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126071+0000", 2024-04-25T20:09:49.387 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0046048979999999996 2024-04-25T20:09:49.387 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.387 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.387 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:49.387 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126141+0000", 2024-04-25T20:09:49.387 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0621e-05 2024-04-25T20:09:49.387 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.388 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.388 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:49.388 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126260+0000", 2024-04-25T20:09:49.388 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000118621 2024-04-25T20:09:49.388 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.388 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.388 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:49.388 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.131270+0000", 2024-04-25T20:09:49.388 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0050096949999999998 2024-04-25T20:09:49.388 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.388 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.388 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:49.388 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132403+0000", 2024-04-25T20:09:49.388 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011332779999999999 2024-04-25T20:09:49.388 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.388 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.388 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:49.389 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132443+0000", 2024-04-25T20:09:49.389 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0175000000000001e-05 2024-04-25T20:09:49.389 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.389 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.389 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:49.389 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132471+0000", 2024-04-25T20:09:49.389 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8399e-05 2024-04-25T20:09:49.389 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.389 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:49.389 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.389 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.389 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.389 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:49.389 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:09:49.389 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 31.184888528999998, 2024-04-25T20:09:49.389 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011071002, 2024-04-25T20:09:49.389 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:49.390 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:49.390 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:49.390 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:49.390 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:49.390 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 141 2024-04-25T20:09:49.390 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.390 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:49.390 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.390 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:49.390 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:09:49.390 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.390 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.390 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.390 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:49.390 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:09:49.390 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.391 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.391 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.391 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:49.391 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121470+0000", 2024-04-25T20:09:49.391 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:49.391 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.391 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.391 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:49.391 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121473+0000", 2024-04-25T20:09:49.391 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.01e-06 2024-04-25T20:09:49.391 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.391 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.391 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:49.391 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121478+0000", 2024-04-25T20:09:49.391 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.515e-06 2024-04-25T20:09:49.391 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.392 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.392 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:49.392 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121480+0000", 2024-04-25T20:09:49.392 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5749999999999999e-06 2024-04-25T20:09:49.392 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.392 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.392 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:49.392 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126813+0000", 2024-04-25T20:09:49.392 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0053320399999999997 2024-04-25T20:09:49.392 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.392 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.392 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:49.392 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126854+0000", 2024-04-25T20:09:49.392 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1582999999999998e-05 2024-04-25T20:09:49.392 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.393 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.393 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:49.393 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126923+0000", 2024-04-25T20:09:49.393 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8544e-05 2024-04-25T20:09:49.393 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.393 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.393 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:49.393 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.131573+0000", 2024-04-25T20:09:49.393 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0046504659999999998 2024-04-25T20:09:49.393 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.393 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.393 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:49.393 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132478+0000", 2024-04-25T20:09:49.393 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00090442400000000003 2024-04-25T20:09:49.393 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.393 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.393 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:49.394 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132513+0000", 2024-04-25T20:09:49.394 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5204999999999999e-05 2024-04-25T20:09:49.394 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.394 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.394 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:49.394 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132542+0000", 2024-04-25T20:09:49.394 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.904e-05 2024-04-25T20:09:49.394 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.394 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:49.394 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.394 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.394 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.394 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:156 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:49.394 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:09:49.394 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 30.687064234000001, 2024-04-25T20:09:49.395 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090072280000000008, 2024-04-25T20:09:49.395 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:49.395 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:49.395 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:49.395 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:49.395 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:49.395 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 156 2024-04-25T20:09:49.395 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.395 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:49.395 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.395 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:49.395 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:09:49.395 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.395 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.395 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.395 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:49.396 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:09:49.396 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.396 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.396 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.396 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:49.396 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:09:49.396 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:49.396 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.396 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.396 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:49.396 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619297+0000", 2024-04-25T20:09:49.396 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2979999999999999e-06 2024-04-25T20:09:49.396 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.396 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.396 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:49.396 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619300+0000", 2024-04-25T20:09:49.397 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.371e-06 2024-04-25T20:09:49.397 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.397 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.397 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:49.397 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619304+0000", 2024-04-25T20:09:49.397 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.304e-06 2024-04-25T20:09:49.397 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.397 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.397 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:49.397 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.622910+0000", 2024-04-25T20:09:49.397 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0036065049999999999 2024-04-25T20:09:49.397 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.397 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.397 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:49.397 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.622990+0000", 2024-04-25T20:09:49.397 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.9722999999999995e-05 2024-04-25T20:09:49.398 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.398 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.398 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:49.398 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623128+0000", 2024-04-25T20:09:49.398 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00013818399999999999 2024-04-25T20:09:49.398 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.398 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.398 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:49.398 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625155+0000", 2024-04-25T20:09:49.398 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0020267979999999998 2024-04-25T20:09:49.398 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.398 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.398 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:49.398 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628225+0000", 2024-04-25T20:09:49.399 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0030705680000000001 2024-04-25T20:09:49.399 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.399 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.399 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:49.399 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628268+0000", 2024-04-25T20:09:49.399 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.2404000000000003e-05 2024-04-25T20:09:49.399 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.399 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.399 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:49.399 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628302+0000", 2024-04-25T20:09:49.399 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4439e-05 2024-04-25T20:09:49.399 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.399 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:49.399 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.399 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.399 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.400 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:157 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:49.400 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:09:49.400 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 30.687052134999998, 2024-04-25T20:09:49.400 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090534889999999996, 2024-04-25T20:09:49.400 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:49.400 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:49.400 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:49.400 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:49.400 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:49.400 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 157 2024-04-25T20:09:49.400 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.400 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:49.400 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.400 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:49.400 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:09:49.400 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.401 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.401 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.401 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:49.401 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:09:49.401 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.401 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.401 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.401 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:49.401 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:09:49.401 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:49.401 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.401 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.401 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:49.401 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619309+0000", 2024-04-25T20:09:49.401 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.841e-06 2024-04-25T20:09:49.401 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.401 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.402 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:49.402 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619312+0000", 2024-04-25T20:09:49.402 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.084e-06 2024-04-25T20:09:49.402 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.402 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.402 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:49.402 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619314+0000", 2024-04-25T20:09:49.402 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.3860000000000001e-06 2024-04-25T20:09:49.402 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.402 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.402 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:49.402 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623718+0000", 2024-04-25T20:09:49.402 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0044038050000000002 2024-04-25T20:09:49.402 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.403 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.403 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:49.403 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623791+0000", 2024-04-25T20:09:49.403 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3181000000000003e-05 2024-04-25T20:09:49.403 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.403 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.403 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:49.403 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623922+0000", 2024-04-25T20:09:49.403 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00013117000000000001 2024-04-25T20:09:49.403 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.403 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.403 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:49.403 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625187+0000", 2024-04-25T20:09:49.403 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001264961 2024-04-25T20:09:49.403 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.404 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.404 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:49.404 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628308+0000", 2024-04-25T20:09:49.404 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.003120739 2024-04-25T20:09:49.404 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.404 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.404 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:49.404 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628331+0000", 2024-04-25T20:09:49.404 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2802999999999998e-05 2024-04-25T20:09:49.404 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.404 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.404 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:49.404 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628361+0000", 2024-04-25T20:09:49.404 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9760999999999998e-05 2024-04-25T20:09:49.404 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.404 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:49.405 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.405 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.405 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.405 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:158 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:49.405 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:09:49.405 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 30.687041688000001, 2024-04-25T20:09:49.405 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0091076409999999997, 2024-04-25T20:09:49.405 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:49.405 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:49.405 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:49.405 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:49.405 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:49.405 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 158 2024-04-25T20:09:49.405 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.405 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:49.405 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.406 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:49.406 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:09:49.406 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.406 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.406 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.406 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:49.406 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:09:49.406 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.406 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.406 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.406 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:49.406 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619317+0000", 2024-04-25T20:09:49.406 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967296 2024-04-25T20:09:49.406 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.406 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.407 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:49.407 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619364+0000", 2024-04-25T20:09:49.407 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.6572000000000001e-05 2024-04-25T20:09:49.407 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.407 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.407 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:49.407 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619371+0000", 2024-04-25T20:09:49.407 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3860000000000001e-06 2024-04-25T20:09:49.407 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.407 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.407 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:49.407 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619382+0000", 2024-04-25T20:09:49.407 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0173999999999999e-05 2024-04-25T20:09:49.407 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.407 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.407 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:49.408 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624216+0000", 2024-04-25T20:09:49.408 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048340350000000004 2024-04-25T20:09:49.408 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.408 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.408 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:49.408 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624260+0000", 2024-04-25T20:09:49.408 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4558999999999998e-05 2024-04-25T20:09:49.408 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.408 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.408 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:49.408 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624333+0000", 2024-04-25T20:09:49.408 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2744999999999996e-05 2024-04-25T20:09:49.408 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.408 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.408 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:49.408 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625465+0000", 2024-04-25T20:09:49.408 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001132454 2024-04-25T20:09:49.409 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.409 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.409 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:49.409 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628365+0000", 2024-04-25T20:09:49.409 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0028994950000000002 2024-04-25T20:09:49.409 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.409 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.409 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:49.409 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628386+0000", 2024-04-25T20:09:49.409 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.1367000000000001e-05 2024-04-25T20:09:49.409 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.409 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.409 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:49.409 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628425+0000", 2024-04-25T20:09:49.409 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.9087e-05 2024-04-25T20:09:49.409 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.410 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:49.410 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.410 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.410 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.410 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:159 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:49.410 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:09:49.410 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 30.686973035000001, 2024-04-25T20:09:49.410 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090948999999999995, 2024-04-25T20:09:49.410 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:49.410 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:49.410 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:49.410 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:49.410 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:49.410 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 159 2024-04-25T20:09:49.410 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.410 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:49.411 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.411 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:49.411 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:09:49.411 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.411 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.411 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.411 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:49.411 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:09:49.411 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:49.411 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.411 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.411 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:49.411 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:09:49.411 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:49.411 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.411 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.411 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:49.412 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619389+0000", 2024-04-25T20:09:49.412 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.734e-06 2024-04-25T20:09:49.412 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.412 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.412 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:49.412 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619392+0000", 2024-04-25T20:09:49.412 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6219999999999998e-06 2024-04-25T20:09:49.412 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.412 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.412 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:49.412 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619403+0000", 2024-04-25T20:09:49.412 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0987e-05 2024-04-25T20:09:49.412 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.412 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.412 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:49.412 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624667+0000", 2024-04-25T20:09:49.413 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0052633970000000004 2024-04-25T20:09:49.413 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.413 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.413 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:49.413 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624741+0000", 2024-04-25T20:09:49.413 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.4583000000000001e-05 2024-04-25T20:09:49.413 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.413 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.413 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:49.413 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624857+0000", 2024-04-25T20:09:49.413 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000116118 2024-04-25T20:09:49.413 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.413 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.413 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:49.413 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.626013+0000", 2024-04-25T20:09:49.413 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001155473 2024-04-25T20:09:49.413 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.414 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.414 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:49.414 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628432+0000", 2024-04-25T20:09:49.414 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0024189379999999998 2024-04-25T20:09:49.414 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.414 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.414 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:49.414 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628454+0000", 2024-04-25T20:09:49.414 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2059999999999999e-05 2024-04-25T20:09:49.414 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:49.414 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:49.414 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:49.414 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628481+0000", 2024-04-25T20:09:49.414 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.7300999999999998e-05 2024-04-25T20:09:49.414 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.415 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:49.415 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.415 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:49.415 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:49.415 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:49.416 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:09:49.416 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:09:49.417 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:09:49.417 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:09:49.417 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T20:09:49.418 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:09:49.421 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:09:49.421 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:49.416+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:49.517 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:49.516+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:49.565 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:49.565 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:49.574 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T20:09:49.618 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:49.618+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:49.680 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:49.681 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:49.689 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T20:09:49.731 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:49.731+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:49.819 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:49.819+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:49.836 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:49.836 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:49.844 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T20:09:49.920 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:49.919+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:49.950 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:49.950 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:49.959 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T20:09:50.022 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:50.021+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:50.065 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:50.065 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:50.073 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T20:09:50.122 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:50.121+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:50.217 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-25T20:09:50.222 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:50.223 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:50.223 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:50.223+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:50.232 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T20:09:50.323 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:50.323+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:50.338 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6, 7] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-25T20:09:50.338 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T20:09:50.338 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-04-25T20:09:50.338 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 0.572445 2024-04-25T20:09:50.338 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 0.5724451245565377 2024-04-25T20:09:50.382 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:50.382 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:50.392 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T20:09:50.436 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:50.436+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:50.502 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:50.502 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:50.511 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T20:09:50.552 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:50.552+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:50.627 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:50.626+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:50.657 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:50.657 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:50.666 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T20:09:50.728 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:50.727+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:50.812 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:50.812 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:50.821 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T20:09:50.864 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:50.864+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:50.926 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:50.926 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:50.930 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:50.929+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:50.935 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T20:09:51.030 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:51.030+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:51.081 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:51.082 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:51.090 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T20:09:51.135 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:51.134+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:51.206 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:51.206 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:51.216 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T20:09:51.218 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is set 2024-04-25T20:09:51.223 INFO:teuthology.orchestra.run.smithi039.stderr:set osd.4 primary-affinity to 0.572445 (8375152) 2024-04-25T20:09:51.231 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:51.231+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:51.333 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:51.332+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:51.358 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:51.358 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:51.367 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T20:09:51.422 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T20:09:51.466 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:51.466+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:51.508 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:51.508 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:51.517 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T20:09:51.534 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:51.534+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:51.544 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:09:51.545 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:09:51.561 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:09:51.562 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:09:51.578 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:09:51.578 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:09:51.594 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:09:51.594 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:09:51.611 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:09:51.611 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:09:51.619 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:51.619 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:51.628 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T20:09:51.629 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:09:51.629 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:09:51.646 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:09:51.646 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:09:51.664 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:09:51.664 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:09:51.671 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:51.671+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:51.731 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:51.731 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:51.731 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:09:51.731 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:09:51.731 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:51.735 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:51.735+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:51.740 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T20:09:51.836 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:51.836+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:51.882 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:51.882 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:09:51.882 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:09:51.882 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:09:51.882 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:51.891 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T20:09:51.937 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:51.937+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:51.991 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:51.992 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:51.992 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:09:51.992 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:52.000 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T20:09:52.038 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:52.038+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:52.101 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:52.101 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:52.101 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:09:52.101 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:09:52.101 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:52.110 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T20:09:52.139 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:52.139+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:52.213 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:52.213 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:09:52.213 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:09:52.213 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:09:52.213 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:52.223 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T20:09:52.240 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:52.240+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:52.324 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:52.324 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:52.324 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:09:52.324 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:52.332 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T20:09:52.376 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:52.375+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:52.434 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:52.434 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:52.434 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:09:52.434 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:09:52.434 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:52.441 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:52.441+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:52.443 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T20:09:52.542 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:52.541+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:52.586 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:52.586 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:09:52.586 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:09:52.586 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [ 2024-04-25T20:09:52.587 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.587 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:09:52.587 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:09:52.587 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 35.802644559999997, 2024-04-25T20:09:52.587 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.34154860100000001, 2024-04-25T20:09:52.587 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:52.587 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:52.587 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:52.587 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:52.587 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:52.587 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 1 2024-04-25T20:09:52.587 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.587 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:52.587 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.587 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:52.587 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:09:52.588 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.588 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.588 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.588 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:52.588 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:09:52.588 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.588 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.588 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.588 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:52.588 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782261+0000", 2024-04-25T20:09:52.588 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999986 2024-04-25T20:09:52.588 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.588 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.588 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:52.588 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782268+0000", 2024-04-25T20:09:52.589 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.8140000000000001e-06 2024-04-25T20:09:52.589 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.589 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.589 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:52.589 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782279+0000", 2024-04-25T20:09:52.589 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0665999999999999e-05 2024-04-25T20:09:52.589 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.589 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.589 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:52.589 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782290+0000", 2024-04-25T20:09:52.589 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.1343e-05 2024-04-25T20:09:52.589 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.589 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.589 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:52.589 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782340+0000", 2024-04-25T20:09:52.590 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.9311999999999998e-05 2024-04-25T20:09:52.590 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.590 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.590 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for peered", 2024-04-25T20:09:52.590 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782348+0000", 2024-04-25T20:09:52.590 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.0690000000000002e-06 2024-04-25T20:09:52.590 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.590 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.590 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:52.590 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120520+0000", 2024-04-25T20:09:52.590 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.33817188100000001 2024-04-25T20:09:52.590 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.590 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.590 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:52.591 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120656+0000", 2024-04-25T20:09:52.591 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000135897 2024-04-25T20:09:52.591 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.591 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.591 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:52.591 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120802+0000", 2024-04-25T20:09:52.591 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00014691199999999999 2024-04-25T20:09:52.591 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.591 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.591 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:52.591 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.122368+0000", 2024-04-25T20:09:52.591 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0015651040000000001 2024-04-25T20:09:52.591 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.591 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.592 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:52.592 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123729+0000", 2024-04-25T20:09:52.592 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0013610499999999999 2024-04-25T20:09:52.592 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.592 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.592 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:52.592 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123769+0000", 2024-04-25T20:09:52.592 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0719000000000001e-05 2024-04-25T20:09:52.592 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.592 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.592 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:52.592 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123811+0000", 2024-04-25T20:09:52.592 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1267999999999999e-05 2024-04-25T20:09:52.592 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.593 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:52.593 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.593 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.593 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.593 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:67 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:09:52.593 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:09:52.593 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 35.373225673, 2024-04-25T20:09:52.593 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.006947651, 2024-04-25T20:09:52.593 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:52.593 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:52.593 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:52.593 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:52.593 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:52.593 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 67 2024-04-25T20:09:52.593 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.594 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:52.594 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.594 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:52.594 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:09:52.594 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.594 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.594 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.594 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:52.594 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:09:52.594 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.594 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.594 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.594 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:52.594 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211680+0000", 2024-04-25T20:09:52.594 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:52.595 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.595 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.595 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:52.595 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211684+0000", 2024-04-25T20:09:52.595 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4249999999999998e-06 2024-04-25T20:09:52.595 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.595 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.595 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:52.595 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211690+0000", 2024-04-25T20:09:52.595 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.7360000000000004e-06 2024-04-25T20:09:52.595 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.595 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.595 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:52.595 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211694+0000", 2024-04-25T20:09:52.595 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4860000000000002e-06 2024-04-25T20:09:52.596 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.596 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.596 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:52.596 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211727+0000", 2024-04-25T20:09:52.596 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3445999999999998e-05 2024-04-25T20:09:52.596 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.596 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.596 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:52.596 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211778+0000", 2024-04-25T20:09:52.596 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1050000000000001e-05 2024-04-25T20:09:52.596 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.596 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.596 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:52.596 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211851+0000", 2024-04-25T20:09:52.597 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2914e-05 2024-04-25T20:09:52.597 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.597 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.597 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:52.597 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213824+0000", 2024-04-25T20:09:52.597 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0019726520000000001 2024-04-25T20:09:52.597 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.597 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.597 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:52.597 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218579+0000", 2024-04-25T20:09:52.597 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0047553159999999999 2024-04-25T20:09:52.597 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.597 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.597 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:52.597 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218607+0000", 2024-04-25T20:09:52.598 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8144999999999999e-05 2024-04-25T20:09:52.598 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.598 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.598 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:52.598 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218629+0000", 2024-04-25T20:09:52.598 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.1044e-05 2024-04-25T20:09:52.598 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.598 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:52.598 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.598 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.598 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.598 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:68 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:09:52.598 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:09:52.598 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 35.373207332, 2024-04-25T20:09:52.598 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012255596000000001, 2024-04-25T20:09:52.599 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:52.599 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:52.599 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:52.599 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:52.599 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:52.599 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 68 2024-04-25T20:09:52.599 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.599 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:52.599 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.599 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:52.599 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:09:52.599 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.599 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.600 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.600 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:52.600 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:09:52.600 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.600 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.600 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.600 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:52.600 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:09:52.600 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:52.600 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.600 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.600 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:52.600 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211701+0000", 2024-04-25T20:09:52.600 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2029999999999999e-06 2024-04-25T20:09:52.600 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.601 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.601 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:52.601 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211705+0000", 2024-04-25T20:09:52.601 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.2320000000000003e-06 2024-04-25T20:09:52.601 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.601 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.601 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:52.601 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211708+0000", 2024-04-25T20:09:52.601 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.6290000000000002e-06 2024-04-25T20:09:52.601 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.601 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.601 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:52.601 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212136+0000", 2024-04-25T20:09:52.601 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00042764099999999998 2024-04-25T20:09:52.601 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.602 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.602 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:52.602 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212212+0000", 2024-04-25T20:09:52.602 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.6315000000000004e-05 2024-04-25T20:09:52.602 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.603 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.603 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:52.603 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212306+0000", 2024-04-25T20:09:52.603 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.3556999999999999e-05 2024-04-25T20:09:52.603 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.603 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.603 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:52.603 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213829+0000", 2024-04-25T20:09:52.603 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001523256 2024-04-25T20:09:52.603 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.603 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.603 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:52.603 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223904+0000", 2024-04-25T20:09:52.603 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010075387 2024-04-25T20:09:52.604 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.604 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.604 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:52.604 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223934+0000", 2024-04-25T20:09:52.604 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9988000000000002e-05 2024-04-25T20:09:52.604 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.604 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.604 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:52.604 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223955+0000", 2024-04-25T20:09:52.604 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0630999999999999e-05 2024-04-25T20:09:52.604 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.604 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:52.604 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.604 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.605 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.605 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:69 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:09:52.605 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:09:52.605 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 35.373185597000003, 2024-04-25T20:09:52.605 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012316863000000001, 2024-04-25T20:09:52.605 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:52.605 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:52.605 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:52.605 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:52.605 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:52.605 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 69 2024-04-25T20:09:52.605 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.605 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:52.605 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.605 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:52.605 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:09:52.605 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.606 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.606 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.606 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:52.606 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:09:52.606 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.606 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.606 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.606 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:52.606 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:09:52.606 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:52.606 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.606 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.606 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:52.606 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211762+0000", 2024-04-25T20:09:52.607 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1703999999999999e-05 2024-04-25T20:09:52.607 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.607 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.607 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:52.607 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211768+0000", 2024-04-25T20:09:52.607 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.3430000000000003e-06 2024-04-25T20:09:52.607 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.607 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.607 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:52.607 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211771+0000", 2024-04-25T20:09:52.607 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.18e-06 2024-04-25T20:09:52.607 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.607 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.607 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:52.607 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212652+0000", 2024-04-25T20:09:52.608 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00088095800000000004 2024-04-25T20:09:52.608 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.608 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.608 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:52.608 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212703+0000", 2024-04-25T20:09:52.608 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.0655999999999997e-05 2024-04-25T20:09:52.608 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.608 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.608 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:52.608 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212781+0000", 2024-04-25T20:09:52.608 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.8852999999999999e-05 2024-04-25T20:09:52.608 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.608 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.608 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:52.608 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.214131+0000", 2024-04-25T20:09:52.609 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0013496039999999999 2024-04-25T20:09:52.609 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.609 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.609 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:52.609 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223986+0000", 2024-04-25T20:09:52.609 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.009854804 2024-04-25T20:09:52.609 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.609 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.609 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:52.609 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.224017+0000", 2024-04-25T20:09:52.609 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.1068999999999997e-05 2024-04-25T20:09:52.609 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.609 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.609 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:52.610 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.224038+0000", 2024-04-25T20:09:52.610 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0965e-05 2024-04-25T20:09:52.610 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.610 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:52.610 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.610 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.610 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.610 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:70 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:09:52.610 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:09:52.610 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 35.373130516000003, 2024-04-25T20:09:52.610 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.015776348999999999, 2024-04-25T20:09:52.610 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:52.610 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:52.611 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:52.611 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:52.611 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:52.611 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 70 2024-04-25T20:09:52.611 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.611 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:52.611 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.611 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:52.611 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:09:52.611 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.611 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.611 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.611 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:52.611 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:09:52.611 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.612 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.612 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.612 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:52.612 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:09:52.612 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:52.612 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.612 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.612 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:52.612 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211778+0000", 2024-04-25T20:09:52.612 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5330000000000001e-06 2024-04-25T20:09:52.612 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.612 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.612 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:52.612 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211782+0000", 2024-04-25T20:09:52.612 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4079999999999998e-06 2024-04-25T20:09:52.612 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.612 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.613 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:52.613 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211786+0000", 2024-04-25T20:09:52.613 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.6190000000000002e-06 2024-04-25T20:09:52.613 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.613 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.613 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:52.613 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213374+0000", 2024-04-25T20:09:52.613 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0015881859999999999 2024-04-25T20:09:52.613 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.613 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.613 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:52.613 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213478+0000", 2024-04-25T20:09:52.613 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000103464 2024-04-25T20:09:52.613 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.613 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.614 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:52.614 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213601+0000", 2024-04-25T20:09:52.614 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00012324300000000001 2024-04-25T20:09:52.614 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.614 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.614 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:52.614 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.214788+0000", 2024-04-25T20:09:52.614 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011868569999999999 2024-04-25T20:09:52.614 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.614 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.614 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:52.614 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227504+0000", 2024-04-25T20:09:52.614 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012715749 2024-04-25T20:09:52.614 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.614 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.615 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:52.615 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227532+0000", 2024-04-25T20:09:52.615 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.7937e-05 2024-04-25T20:09:52.615 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.615 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.615 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:52.615 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227552+0000", 2024-04-25T20:09:52.615 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0772999999999999e-05 2024-04-25T20:09:52.615 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.615 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:52.615 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.615 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.615 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.615 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:131 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:52.615 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:09:52.616 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 34.464120671000003, 2024-04-25T20:09:52.616 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0086158810000000006, 2024-04-25T20:09:52.616 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:52.616 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:52.616 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:52.616 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:52.616 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:52.616 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 131 2024-04-25T20:09:52.616 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.616 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:52.616 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.616 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:52.616 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:09:52.616 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.616 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.617 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.617 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:52.617 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:09:52.617 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.617 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.617 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.617 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:52.617 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120785+0000", 2024-04-25T20:09:52.617 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:52.617 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.617 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.617 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:52.617 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120880+0000", 2024-04-25T20:09:52.617 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.4925000000000003e-05 2024-04-25T20:09:52.617 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.617 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.618 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:52.618 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120891+0000", 2024-04-25T20:09:52.618 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0915e-05 2024-04-25T20:09:52.618 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.618 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.618 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:52.618 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120895+0000", 2024-04-25T20:09:52.618 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.8070000000000001e-06 2024-04-25T20:09:52.618 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.618 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.618 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:52.618 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121640+0000", 2024-04-25T20:09:52.618 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00074468500000000003 2024-04-25T20:09:52.618 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.618 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.619 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:52.619 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121684+0000", 2024-04-25T20:09:52.619 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4452000000000001e-05 2024-04-25T20:09:52.619 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.619 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.619 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:52.619 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121753+0000", 2024-04-25T20:09:52.619 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8508000000000002e-05 2024-04-25T20:09:52.619 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.619 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.619 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:52.619 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123618+0000", 2024-04-25T20:09:52.619 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0018655099999999999 2024-04-25T20:09:52.619 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.619 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.619 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:52.619 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129248+0000", 2024-04-25T20:09:52.620 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0056295540000000002 2024-04-25T20:09:52.620 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.620 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.620 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:52.620 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129368+0000", 2024-04-25T20:09:52.620 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000120244 2024-04-25T20:09:52.620 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.620 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.620 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:52.620 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129402+0000", 2024-04-25T20:09:52.620 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3812000000000001e-05 2024-04-25T20:09:52.620 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.620 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:52.620 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.620 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.621 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.621 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:132 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:52.621 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:09:52.621 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 34.464006460999997, 2024-04-25T20:09:52.621 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0085898169999999996, 2024-04-25T20:09:52.621 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:52.621 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:52.621 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:52.621 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:52.621 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:52.621 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 132 2024-04-25T20:09:52.621 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.621 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:52.621 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.621 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:52.621 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:09:52.621 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.622 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.622 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.622 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:52.622 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:09:52.622 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.622 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.622 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.622 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:52.622 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:09:52.622 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:52.622 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.622 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.622 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:52.622 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120902+0000", 2024-04-25T20:09:52.623 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.6129999999999998e-06 2024-04-25T20:09:52.623 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.623 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.623 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:52.623 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120906+0000", 2024-04-25T20:09:52.623 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6569999999999999e-06 2024-04-25T20:09:52.623 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.623 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.623 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:52.623 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120909+0000", 2024-04-25T20:09:52.623 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5550000000000001e-06 2024-04-25T20:09:52.623 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.623 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.623 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:52.623 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122098+0000", 2024-04-25T20:09:52.624 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011898049999999999 2024-04-25T20:09:52.624 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.624 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.624 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:52.624 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122172+0000", 2024-04-25T20:09:52.624 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3917000000000004e-05 2024-04-25T20:09:52.624 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.624 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.624 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:52.624 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122288+0000", 2024-04-25T20:09:52.624 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00011566 2024-04-25T20:09:52.624 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.624 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.624 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:52.624 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124519+0000", 2024-04-25T20:09:52.625 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0022308359999999999 2024-04-25T20:09:52.625 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.625 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.625 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:52.625 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129414+0000", 2024-04-25T20:09:52.625 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048953720000000003 2024-04-25T20:09:52.625 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.625 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.625 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:52.625 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129458+0000", 2024-04-25T20:09:52.625 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.3426000000000001e-05 2024-04-25T20:09:52.625 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.625 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.625 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:52.625 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129490+0000", 2024-04-25T20:09:52.626 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2329999999999997e-05 2024-04-25T20:09:52.626 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.626 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:52.626 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.626 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.626 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.626 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:133 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:52.626 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:09:52.626 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 34.463952591999998, 2024-04-25T20:09:52.626 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0086066270000000004, 2024-04-25T20:09:52.626 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:52.626 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:52.626 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:52.627 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:52.627 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:52.627 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 133 2024-04-25T20:09:52.627 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.627 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:52.627 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.627 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:52.627 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:09:52.627 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.627 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.627 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.627 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:52.627 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:09:52.627 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.627 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.628 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.628 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:52.628 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:09:52.628 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:52.628 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.628 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.628 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:52.628 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120956+0000", 2024-04-25T20:09:52.628 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5289999999999998e-06 2024-04-25T20:09:52.628 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.628 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.628 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:52.628 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120962+0000", 2024-04-25T20:09:52.628 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.9490000000000001e-06 2024-04-25T20:09:52.628 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.628 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.629 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:52.629 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120965+0000", 2024-04-25T20:09:52.629 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.1860000000000001e-06 2024-04-25T20:09:52.629 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.629 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.629 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:52.629 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122666+0000", 2024-04-25T20:09:52.629 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0017004500000000001 2024-04-25T20:09:52.629 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.629 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.629 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:52.629 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122741+0000", 2024-04-25T20:09:52.629 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.5297999999999996e-05 2024-04-25T20:09:52.629 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.629 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.630 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:52.630 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122858+0000", 2024-04-25T20:09:52.630 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00011723899999999999 2024-04-25T20:09:52.630 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.630 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.630 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:52.630 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124625+0000", 2024-04-25T20:09:52.630 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0017670100000000001 2024-04-25T20:09:52.630 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.630 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.630 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:52.630 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129497+0000", 2024-04-25T20:09:52.630 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048713580000000001 2024-04-25T20:09:52.631 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.631 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.631 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:52.631 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129532+0000", 2024-04-25T20:09:52.631 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5704999999999998e-05 2024-04-25T20:09:52.631 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.631 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.631 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:52.631 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129561+0000", 2024-04-25T20:09:52.631 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8283e-05 2024-04-25T20:09:52.631 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.631 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:52.631 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.631 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.631 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.632 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:134 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:52.632 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:09:52.632 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 34.463850393999998, 2024-04-25T20:09:52.632 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0085771179999999999, 2024-04-25T20:09:52.632 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:52.632 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:52.632 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:52.632 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:52.632 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:52.632 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 134 2024-04-25T20:09:52.632 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.632 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:52.632 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.632 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:52.632 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:09:52.633 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.633 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.633 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.633 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:52.633 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:09:52.633 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.633 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.633 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.633 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:52.633 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:09:52.633 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:52.633 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.633 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.633 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:52.634 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121157+0000", 2024-04-25T20:09:52.634 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00010104599999999999 2024-04-25T20:09:52.634 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.634 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.634 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:52.634 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121189+0000", 2024-04-25T20:09:52.634 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2589000000000003e-05 2024-04-25T20:09:52.634 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.634 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.634 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:52.634 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121195+0000", 2024-04-25T20:09:52.634 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.083e-06 2024-04-25T20:09:52.634 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.634 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.635 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:52.635 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123136+0000", 2024-04-25T20:09:52.635 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0019409780000000001 2024-04-25T20:09:52.635 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.635 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.635 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:52.635 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123181+0000", 2024-04-25T20:09:52.635 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4267999999999997e-05 2024-04-25T20:09:52.635 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.635 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.635 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:52.635 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123247+0000", 2024-04-25T20:09:52.635 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.6086000000000004e-05 2024-04-25T20:09:52.635 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.635 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.636 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:52.636 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125639+0000", 2024-04-25T20:09:52.636 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.002392 2024-04-25T20:09:52.636 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.636 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.636 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:52.636 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129567+0000", 2024-04-25T20:09:52.636 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0039282170000000003 2024-04-25T20:09:52.636 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.636 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.636 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:52.636 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129605+0000", 2024-04-25T20:09:52.636 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.7660999999999997e-05 2024-04-25T20:09:52.636 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.636 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.636 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:52.637 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129633+0000", 2024-04-25T20:09:52.637 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8649999999999998e-05 2024-04-25T20:09:52.637 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.637 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:52.637 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.637 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.637 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.637 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:135 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:52.637 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:09:52.637 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 34.463691388000001, 2024-04-25T20:09:52.637 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010898453000000001, 2024-04-25T20:09:52.637 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:52.637 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:52.637 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:52.637 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:52.637 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:52.638 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 135 2024-04-25T20:09:52.638 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.638 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:52.638 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.638 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:52.638 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:09:52.638 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.638 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.638 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.638 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:52.638 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:09:52.638 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.639 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.639 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.639 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:52.639 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121214+0000", 2024-04-25T20:09:52.639 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:52.639 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.639 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.639 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:52.639 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121218+0000", 2024-04-25T20:09:52.639 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6619999999999998e-06 2024-04-25T20:09:52.639 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.639 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.639 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:52.639 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121223+0000", 2024-04-25T20:09:52.639 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1070000000000004e-06 2024-04-25T20:09:52.639 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.640 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.640 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:52.640 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121227+0000", 2024-04-25T20:09:52.640 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2820000000000001e-06 2024-04-25T20:09:52.640 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.640 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.640 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:52.640 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123649+0000", 2024-04-25T20:09:52.640 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0024221970000000001 2024-04-25T20:09:52.640 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.640 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.640 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:52.640 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123723+0000", 2024-04-25T20:09:52.640 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.4386000000000002e-05 2024-04-25T20:09:52.640 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.641 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.641 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:52.641 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123844+0000", 2024-04-25T20:09:52.641 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000121249 2024-04-25T20:09:52.641 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.641 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.641 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:52.641 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125670+0000", 2024-04-25T20:09:52.641 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001825901 2024-04-25T20:09:52.641 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.641 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.641 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:52.641 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132043+0000", 2024-04-25T20:09:52.641 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0063724539999999996 2024-04-25T20:09:52.642 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.642 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.642 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:52.642 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132084+0000", 2024-04-25T20:09:52.642 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0963000000000001e-05 2024-04-25T20:09:52.642 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.642 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.642 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:52.642 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132114+0000", 2024-04-25T20:09:52.642 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9927e-05 2024-04-25T20:09:52.642 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.642 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:52.642 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.642 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.643 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.643 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:136 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:52.643 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:09:52.643 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 34.463675862000002, 2024-04-25T20:09:52.643 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010952513000000001, 2024-04-25T20:09:52.643 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:52.643 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:52.643 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:52.643 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:52.643 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:52.643 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 136 2024-04-25T20:09:52.643 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.643 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:52.643 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.643 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:52.644 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:09:52.644 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.644 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.644 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.644 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:52.644 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:09:52.644 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.644 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.644 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.644 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:52.644 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121230+0000", 2024-04-25T20:09:52.644 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:52.644 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.644 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.644 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:52.645 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121236+0000", 2024-04-25T20:09:52.645 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.7320000000000003e-06 2024-04-25T20:09:52.645 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.645 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.645 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:52.645 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121242+0000", 2024-04-25T20:09:52.645 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.7810000000000002e-06 2024-04-25T20:09:52.645 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.645 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.645 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:52.645 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121246+0000", 2024-04-25T20:09:52.645 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.639e-06 2024-04-25T20:09:52.645 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.645 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.645 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:52.646 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124150+0000", 2024-04-25T20:09:52.646 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.002903816 2024-04-25T20:09:52.646 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.646 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.646 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:52.646 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124195+0000", 2024-04-25T20:09:52.646 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.5042e-05 2024-04-25T20:09:52.646 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.646 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.646 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:52.646 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124266+0000", 2024-04-25T20:09:52.646 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0234999999999994e-05 2024-04-25T20:09:52.646 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.646 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.646 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:52.647 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125711+0000", 2024-04-25T20:09:52.647 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0014458800000000001 2024-04-25T20:09:52.647 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.647 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.647 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:52.647 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132120+0000", 2024-04-25T20:09:52.647 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0064084399999999996 2024-04-25T20:09:52.647 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.647 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.647 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:52.647 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132155+0000", 2024-04-25T20:09:52.647 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4938000000000003e-05 2024-04-25T20:09:52.647 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.647 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.647 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:52.647 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132183+0000", 2024-04-25T20:09:52.648 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8360999999999998e-05 2024-04-25T20:09:52.648 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.648 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:52.648 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.648 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.648 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.648 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:52.648 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:09:52.648 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 34.463626130000002, 2024-04-25T20:09:52.648 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.01097684, 2024-04-25T20:09:52.648 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:52.648 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:52.648 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:52.648 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:52.649 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:52.649 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 137 2024-04-25T20:09:52.649 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.649 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:52.649 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.649 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:52.649 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:09:52.649 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.649 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.649 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.649 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:52.649 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:09:52.649 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.649 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.649 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.649 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:52.650 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:09:52.650 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:52.650 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.650 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.650 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:52.650 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121328+0000", 2024-04-25T20:09:52.650 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.7908999999999998e-05 2024-04-25T20:09:52.650 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.650 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.650 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:52.650 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121337+0000", 2024-04-25T20:09:52.650 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.9369999999999993e-06 2024-04-25T20:09:52.650 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.651 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.651 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:52.651 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121343+0000", 2024-04-25T20:09:52.651 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.8089999999999998e-06 2024-04-25T20:09:52.651 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.651 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.651 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:52.651 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124656+0000", 2024-04-25T20:09:52.651 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0033133839999999999 2024-04-25T20:09:52.651 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.651 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.651 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:52.651 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124730+0000", 2024-04-25T20:09:52.651 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3565000000000005e-05 2024-04-25T20:09:52.651 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.652 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.652 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:52.652 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124846+0000", 2024-04-25T20:09:52.652 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000116416 2024-04-25T20:09:52.652 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.652 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.652 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:52.652 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126756+0000", 2024-04-25T20:09:52.652 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001909991 2024-04-25T20:09:52.652 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.652 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.652 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:52.652 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132190+0000", 2024-04-25T20:09:52.652 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0054343880000000001 2024-04-25T20:09:52.653 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.653 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.653 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:52.653 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132227+0000", 2024-04-25T20:09:52.653 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6875000000000002e-05 2024-04-25T20:09:52.653 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.653 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.653 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:52.653 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132257+0000", 2024-04-25T20:09:52.653 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.0046e-05 2024-04-25T20:09:52.653 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.653 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:52.653 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.653 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.653 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.654 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:52.654 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:09:52.654 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 34.463556171999997, 2024-04-25T20:09:52.654 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010978653999999999, 2024-04-25T20:09:52.654 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:52.654 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:52.654 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:52.654 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:52.654 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:52.654 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 138 2024-04-25T20:09:52.654 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.654 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:52.654 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.654 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:52.654 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:09:52.655 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.655 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.655 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.655 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:52.655 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:09:52.655 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.655 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.655 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.655 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:52.655 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:09:52.655 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:52.655 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.655 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.655 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:52.655 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121355+0000", 2024-04-25T20:09:52.656 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.9869999999999999e-06 2024-04-25T20:09:52.656 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.656 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.656 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:52.656 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121361+0000", 2024-04-25T20:09:52.656 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.4620000000000001e-06 2024-04-25T20:09:52.656 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.656 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.656 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:52.656 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121373+0000", 2024-04-25T20:09:52.656 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.2035000000000001e-05 2024-04-25T20:09:52.656 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.656 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.656 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:52.656 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125267+0000", 2024-04-25T20:09:52.657 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0038939230000000001 2024-04-25T20:09:52.657 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.657 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.657 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:52.657 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125309+0000", 2024-04-25T20:09:52.657 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1690999999999998e-05 2024-04-25T20:09:52.657 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.657 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.657 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:52.657 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125385+0000", 2024-04-25T20:09:52.657 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.6162999999999996e-05 2024-04-25T20:09:52.657 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.657 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.657 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:52.657 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126798+0000", 2024-04-25T20:09:52.658 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0014124680000000001 2024-04-25T20:09:52.658 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.658 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.658 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:52.658 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132264+0000", 2024-04-25T20:09:52.658 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0054661689999999999 2024-04-25T20:09:52.658 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.658 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.658 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:52.658 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132299+0000", 2024-04-25T20:09:52.658 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5272999999999999e-05 2024-04-25T20:09:52.658 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.658 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.658 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:52.659 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132329+0000", 2024-04-25T20:09:52.659 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.0080999999999999e-05 2024-04-25T20:09:52.659 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.659 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:52.659 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.659 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.659 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.659 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:52.659 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:09:52.659 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 34.463525597, 2024-04-25T20:09:52.659 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011015853000000001, 2024-04-25T20:09:52.659 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:52.659 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:52.659 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:52.659 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:52.659 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:52.659 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 139 2024-04-25T20:09:52.660 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.660 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:52.660 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.660 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:52.660 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:09:52.660 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.660 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.660 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.660 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:52.660 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:09:52.660 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.660 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.660 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.660 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:52.660 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121380+0000", 2024-04-25T20:09:52.660 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:52.660 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.661 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.661 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:52.661 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121385+0000", 2024-04-25T20:09:52.661 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.092e-06 2024-04-25T20:09:52.661 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.661 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.661 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:52.661 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121390+0000", 2024-04-25T20:09:52.661 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.5990000000000001e-06 2024-04-25T20:09:52.661 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.661 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.661 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:52.661 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121397+0000", 2024-04-25T20:09:52.661 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.6279999999999999e-06 2024-04-25T20:09:52.661 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.662 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.662 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:52.662 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125737+0000", 2024-04-25T20:09:52.662 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0043406010000000004 2024-04-25T20:09:52.662 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.662 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.662 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:52.662 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125781+0000", 2024-04-25T20:09:52.662 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.3958999999999997e-05 2024-04-25T20:09:52.662 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.662 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.662 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:52.662 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125849+0000", 2024-04-25T20:09:52.662 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8071999999999995e-05 2024-04-25T20:09:52.663 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.663 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.663 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:52.663 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.130908+0000", 2024-04-25T20:09:52.663 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0050586509999999999 2024-04-25T20:09:52.663 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.663 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.663 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:52.663 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132334+0000", 2024-04-25T20:09:52.663 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00142621 2024-04-25T20:09:52.663 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.663 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.663 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:52.663 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132369+0000", 2024-04-25T20:09:52.664 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4452000000000002e-05 2024-04-25T20:09:52.664 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.664 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.664 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:52.664 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132397+0000", 2024-04-25T20:09:52.664 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8065e-05 2024-04-25T20:09:52.664 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.664 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:52.664 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.664 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.664 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.664 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:52.664 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:09:52.664 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 34.463504553, 2024-04-25T20:09:52.664 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011069443, 2024-04-25T20:09:52.665 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:52.665 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:52.665 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:52.665 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:52.665 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:52.665 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 140 2024-04-25T20:09:52.665 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.665 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:52.665 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.665 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:52.665 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:09:52.665 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.665 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.665 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.666 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:52.666 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:09:52.666 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.666 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.666 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.666 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:52.666 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:09:52.666 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:52.666 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.666 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.666 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:52.666 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121456+0000", 2024-04-25T20:09:52.666 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.4469000000000003e-05 2024-04-25T20:09:52.666 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.667 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.667 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:52.667 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121462+0000", 2024-04-25T20:09:52.667 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.1890000000000004e-06 2024-04-25T20:09:52.667 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.667 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.667 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:52.667 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121466+0000", 2024-04-25T20:09:52.667 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3909999999999998e-06 2024-04-25T20:09:52.667 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.667 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.667 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:52.667 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126071+0000", 2024-04-25T20:09:52.667 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0046048979999999996 2024-04-25T20:09:52.667 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.668 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.668 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:52.668 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126141+0000", 2024-04-25T20:09:52.668 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0621e-05 2024-04-25T20:09:52.668 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.668 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.668 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:52.668 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126260+0000", 2024-04-25T20:09:52.668 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000118621 2024-04-25T20:09:52.668 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.668 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.668 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:52.668 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.131270+0000", 2024-04-25T20:09:52.668 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0050096949999999998 2024-04-25T20:09:52.669 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.669 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.669 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:52.669 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132403+0000", 2024-04-25T20:09:52.669 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011332779999999999 2024-04-25T20:09:52.669 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.669 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.669 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:52.669 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132443+0000", 2024-04-25T20:09:52.669 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0175000000000001e-05 2024-04-25T20:09:52.669 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.669 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.669 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:52.669 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132471+0000", 2024-04-25T20:09:52.669 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8399e-05 2024-04-25T20:09:52.670 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.670 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:52.670 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.670 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.670 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.670 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:52.670 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:09:52.670 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 34.463435767999997, 2024-04-25T20:09:52.670 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011071002, 2024-04-25T20:09:52.670 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:52.670 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:52.670 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:52.670 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:52.671 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:52.671 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 141 2024-04-25T20:09:52.671 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.671 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:52.671 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.671 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:52.671 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:09:52.671 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.671 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.671 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.671 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:52.671 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:09:52.671 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.671 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.672 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.672 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:52.672 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121470+0000", 2024-04-25T20:09:52.672 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:52.672 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.672 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.672 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:52.672 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121473+0000", 2024-04-25T20:09:52.672 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.01e-06 2024-04-25T20:09:52.672 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.672 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.672 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:52.672 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121478+0000", 2024-04-25T20:09:52.672 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.515e-06 2024-04-25T20:09:52.672 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.672 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.673 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:52.673 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121480+0000", 2024-04-25T20:09:52.673 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5749999999999999e-06 2024-04-25T20:09:52.673 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.673 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.673 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:52.673 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126813+0000", 2024-04-25T20:09:52.673 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0053320399999999997 2024-04-25T20:09:52.673 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.673 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.673 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:52.673 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126854+0000", 2024-04-25T20:09:52.673 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1582999999999998e-05 2024-04-25T20:09:52.673 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.673 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.673 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:52.673 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126923+0000", 2024-04-25T20:09:52.674 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8544e-05 2024-04-25T20:09:52.674 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.674 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.674 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:52.674 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.131573+0000", 2024-04-25T20:09:52.674 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0046504659999999998 2024-04-25T20:09:52.674 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.674 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.674 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:52.674 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132478+0000", 2024-04-25T20:09:52.674 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00090442400000000003 2024-04-25T20:09:52.674 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.674 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.674 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:52.674 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132513+0000", 2024-04-25T20:09:52.674 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5204999999999999e-05 2024-04-25T20:09:52.675 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.675 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.675 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:52.675 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132542+0000", 2024-04-25T20:09:52.675 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.904e-05 2024-04-25T20:09:52.675 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.675 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:52.675 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.675 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.675 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.675 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:156 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:52.675 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:09:52.675 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 33.965611473000003, 2024-04-25T20:09:52.675 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090072280000000008, 2024-04-25T20:09:52.675 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:52.675 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:52.676 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:52.676 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:52.676 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:52.676 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 156 2024-04-25T20:09:52.676 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.676 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:52.676 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.676 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:52.676 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:09:52.676 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.676 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.676 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.676 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:52.676 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:09:52.676 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.677 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.677 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.677 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:52.677 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:09:52.677 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:52.677 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.677 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.677 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:52.677 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619297+0000", 2024-04-25T20:09:52.677 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2979999999999999e-06 2024-04-25T20:09:52.677 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.677 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.677 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:52.677 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619300+0000", 2024-04-25T20:09:52.677 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.371e-06 2024-04-25T20:09:52.677 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.678 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.678 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:52.678 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619304+0000", 2024-04-25T20:09:52.678 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.304e-06 2024-04-25T20:09:52.678 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.678 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.678 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:52.678 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.622910+0000", 2024-04-25T20:09:52.678 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0036065049999999999 2024-04-25T20:09:52.678 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.678 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.678 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:52.678 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.622990+0000", 2024-04-25T20:09:52.678 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.9722999999999995e-05 2024-04-25T20:09:52.678 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.678 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.679 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:52.679 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623128+0000", 2024-04-25T20:09:52.679 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00013818399999999999 2024-04-25T20:09:52.679 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.679 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.679 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:52.679 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625155+0000", 2024-04-25T20:09:52.679 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0020267979999999998 2024-04-25T20:09:52.679 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.679 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.679 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:52.679 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628225+0000", 2024-04-25T20:09:52.679 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0030705680000000001 2024-04-25T20:09:52.679 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.679 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.680 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:52.680 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628268+0000", 2024-04-25T20:09:52.680 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.2404000000000003e-05 2024-04-25T20:09:52.680 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.680 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.680 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:52.680 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628302+0000", 2024-04-25T20:09:52.680 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4439e-05 2024-04-25T20:09:52.680 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.680 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:52.680 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.680 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.680 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.680 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:157 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:52.681 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:09:52.681 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 33.965599374, 2024-04-25T20:09:52.681 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090534889999999996, 2024-04-25T20:09:52.681 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:52.681 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:52.681 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:52.681 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:52.681 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:52.681 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 157 2024-04-25T20:09:52.681 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.681 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:52.681 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.681 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:52.681 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:09:52.682 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.682 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.682 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.682 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:52.682 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:09:52.682 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.682 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.682 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.682 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:52.682 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:09:52.682 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:52.682 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.682 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.682 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:52.683 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619309+0000", 2024-04-25T20:09:52.683 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.841e-06 2024-04-25T20:09:52.683 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.683 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.683 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:52.683 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619312+0000", 2024-04-25T20:09:52.683 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.084e-06 2024-04-25T20:09:52.683 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.683 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.683 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:52.683 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619314+0000", 2024-04-25T20:09:52.683 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.3860000000000001e-06 2024-04-25T20:09:52.683 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.683 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.683 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:52.683 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623718+0000", 2024-04-25T20:09:52.684 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0044038050000000002 2024-04-25T20:09:52.684 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.684 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.684 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:52.684 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623791+0000", 2024-04-25T20:09:52.684 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3181000000000003e-05 2024-04-25T20:09:52.684 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.684 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.684 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:52.684 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623922+0000", 2024-04-25T20:09:52.684 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00013117000000000001 2024-04-25T20:09:52.684 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.684 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.684 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:52.684 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625187+0000", 2024-04-25T20:09:52.685 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001264961 2024-04-25T20:09:52.685 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.685 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.685 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:52.685 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628308+0000", 2024-04-25T20:09:52.685 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.003120739 2024-04-25T20:09:52.685 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.685 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.685 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:52.685 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628331+0000", 2024-04-25T20:09:52.685 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2802999999999998e-05 2024-04-25T20:09:52.685 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.685 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.685 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:52.685 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628361+0000", 2024-04-25T20:09:52.685 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9760999999999998e-05 2024-04-25T20:09:52.685 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.686 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:52.686 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.686 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.686 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.686 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:158 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:52.686 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:09:52.686 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 33.965588926999999, 2024-04-25T20:09:52.686 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0091076409999999997, 2024-04-25T20:09:52.686 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:52.686 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:52.686 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:52.686 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:52.686 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:52.686 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 158 2024-04-25T20:09:52.686 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.687 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:52.687 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.687 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:52.687 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:09:52.687 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.687 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.687 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.687 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:52.687 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:09:52.687 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.687 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.687 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.687 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:52.687 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619317+0000", 2024-04-25T20:09:52.687 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967296 2024-04-25T20:09:52.688 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.688 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.688 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:52.688 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619364+0000", 2024-04-25T20:09:52.688 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.6572000000000001e-05 2024-04-25T20:09:52.688 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.689 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.689 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:52.689 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619371+0000", 2024-04-25T20:09:52.689 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3860000000000001e-06 2024-04-25T20:09:52.690 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.690 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.690 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:52.690 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619382+0000", 2024-04-25T20:09:52.690 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0173999999999999e-05 2024-04-25T20:09:52.690 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.690 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.690 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:52.690 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624216+0000", 2024-04-25T20:09:52.690 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048340350000000004 2024-04-25T20:09:52.690 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.690 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.690 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:52.690 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624260+0000", 2024-04-25T20:09:52.690 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4558999999999998e-05 2024-04-25T20:09:52.691 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.691 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.691 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:52.691 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624333+0000", 2024-04-25T20:09:52.691 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2744999999999996e-05 2024-04-25T20:09:52.691 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.691 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.691 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:52.691 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625465+0000", 2024-04-25T20:09:52.691 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001132454 2024-04-25T20:09:52.691 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.691 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.691 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:52.691 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628365+0000", 2024-04-25T20:09:52.692 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0028994950000000002 2024-04-25T20:09:52.692 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.692 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.692 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:52.692 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628386+0000", 2024-04-25T20:09:52.692 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.1367000000000001e-05 2024-04-25T20:09:52.692 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.692 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.692 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:52.692 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628425+0000", 2024-04-25T20:09:52.692 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.9087e-05 2024-04-25T20:09:52.692 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.692 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:52.692 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.692 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.693 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.693 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:159 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:09:52.693 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:09:52.693 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 33.965520273999999, 2024-04-25T20:09:52.693 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090948999999999995, 2024-04-25T20:09:52.693 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:09:52.693 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:52.693 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:09:52.693 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:09:52.693 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:09:52.693 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 159 2024-04-25T20:09:52.693 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.693 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:09:52.693 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.693 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:09:52.693 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:09:52.694 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.694 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.694 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.694 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:09:52.694 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:09:52.694 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:09:52.694 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.694 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.694 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:09:52.694 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:09:52.694 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:52.694 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.694 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.695 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:09:52.695 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619389+0000", 2024-04-25T20:09:52.695 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.734e-06 2024-04-25T20:09:52.695 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.695 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.695 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:09:52.695 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619392+0000", 2024-04-25T20:09:52.695 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6219999999999998e-06 2024-04-25T20:09:52.695 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.695 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.695 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:09:52.695 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619403+0000", 2024-04-25T20:09:52.695 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0987e-05 2024-04-25T20:09:52.695 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.695 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.695 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:09:52.695 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624667+0000", 2024-04-25T20:09:52.696 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0052633970000000004 2024-04-25T20:09:52.696 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.696 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.696 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:09:52.696 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624741+0000", 2024-04-25T20:09:52.696 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.4583000000000001e-05 2024-04-25T20:09:52.696 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.696 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.696 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:09:52.696 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624857+0000", 2024-04-25T20:09:52.696 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000116118 2024-04-25T20:09:52.696 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.696 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.696 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:09:52.696 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.626013+0000", 2024-04-25T20:09:52.696 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001155473 2024-04-25T20:09:52.696 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.697 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.697 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:09:52.697 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628432+0000", 2024-04-25T20:09:52.697 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0024189379999999998 2024-04-25T20:09:52.697 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.697 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.697 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:09:52.697 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628454+0000", 2024-04-25T20:09:52.697 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2059999999999999e-05 2024-04-25T20:09:52.697 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:09:52.697 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:09:52.697 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:09:52.697 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628481+0000", 2024-04-25T20:09:52.697 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.7300999999999998e-05 2024-04-25T20:09:52.697 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.697 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:52.698 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.698 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:09:52.698 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:09:52.698 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:52.699 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T20:09:52.700 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:52.699+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:52.800 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:52.800+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:52.818 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:52.818 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:52.818 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:09:52.818 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:52.828 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T20:09:52.913 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:52.913+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:52.976 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:52.976 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:52.976 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:09:52.976 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:09:52.976 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:52.985 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T20:09:53.001 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:53.001+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:53.102 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:53.102+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:53.131 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:53.131 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:09:53.131 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:09:53.131 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:09:53.131 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:53.140 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T20:09:53.224 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:53.224+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:53.233 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-25T20:09:53.288 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:53.288 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:53.288 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:09:53.288 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:53.297 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T20:09:53.341 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:53.341+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:53.404 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:53.404+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:53.405 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:53.405 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:53.405 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:09:53.405 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:09:53.405 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:53.414 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T20:09:53.506 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:53.505+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:53.562 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:53.562 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:09:53.562 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:09:53.562 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-04-25T20:09:53.562 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.562 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:30 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'16, mlcod=16'14)", 2024-04-25T20:09:53.562 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:09:53.562 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 36.393584273999998, 2024-04-25T20:09:53.562 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0054087670000000001, 2024-04-25T20:09:53.562 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:53.562 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:53.563 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:53.563 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.563 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:53.563 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:09:53.563 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.563 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.563 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.563 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:53.563 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:09:53.563 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.563 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.563 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.563 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:53.563 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167142+0000", 2024-04-25T20:09:53.563 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999976 2024-04-25T20:09:53.563 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.564 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.564 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:53.564 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167162+0000", 2024-04-25T20:09:53.564 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0109e-05 2024-04-25T20:09:53.564 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.564 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.564 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:53.564 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167188+0000", 2024-04-25T20:09:53.564 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6146e-05 2024-04-25T20:09:53.564 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.564 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.564 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:53.564 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167199+0000", 2024-04-25T20:09:53.564 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0767e-05 2024-04-25T20:09:53.564 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.564 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.565 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:53.565 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167256+0000", 2024-04-25T20:09:53.565 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7392000000000002e-05 2024-04-25T20:09:53.565 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.565 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.565 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:53.565 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167283+0000", 2024-04-25T20:09:53.565 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6577e-05 2024-04-25T20:09:53.565 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.565 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.565 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:53.565 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.172510+0000", 2024-04-25T20:09:53.565 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0052276759999999997 2024-04-25T20:09:53.565 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.565 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.565 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:53.566 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.172553+0000", 2024-04-25T20:09:53.566 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2444999999999998e-05 2024-04-25T20:09:53.566 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.566 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:53.566 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.566 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.566 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.566 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:31 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-04-25T20:09:53.566 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:09:53.566 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 36.392968195999998, 2024-04-25T20:09:53.566 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0055055479999999999, 2024-04-25T20:09:53.566 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:53.566 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:53.567 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:53.567 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.567 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:53.567 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:09:53.567 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.567 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.567 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.567 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:53.567 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:09:53.567 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.567 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.567 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.567 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:53.567 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:09:53.567 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:53.567 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.567 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.568 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:53.568 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167766+0000", 2024-04-25T20:09:53.568 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.122e-06 2024-04-25T20:09:53.568 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.568 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.568 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:53.568 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167776+0000", 2024-04-25T20:09:53.568 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0515e-05 2024-04-25T20:09:53.568 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.568 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.568 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:53.568 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167782+0000", 2024-04-25T20:09:53.568 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1149999999999999e-06 2024-04-25T20:09:53.568 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.568 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.568 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:53.569 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167837+0000", 2024-04-25T20:09:53.569 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4763999999999999e-05 2024-04-25T20:09:53.569 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.569 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.569 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:53.569 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167855+0000", 2024-04-25T20:09:53.569 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7589e-05 2024-04-25T20:09:53.569 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.569 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.569 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:53.569 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173221+0000", 2024-04-25T20:09:53.569 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.005365903 2024-04-25T20:09:53.569 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.569 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.570 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:53.570 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173266+0000", 2024-04-25T20:09:53.570 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5059999999999999e-05 2024-04-25T20:09:53.570 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.570 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:53.570 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.570 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.570 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.570 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:32 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-04-25T20:09:53.570 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:09:53.570 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 36.392593011000002, 2024-04-25T20:09:53.570 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0051558239999999998, 2024-04-25T20:09:53.570 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:53.571 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:53.571 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:53.571 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.571 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:53.571 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:09:53.571 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.571 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.571 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.571 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:53.571 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:09:53.571 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.571 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.571 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.571 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:53.571 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:09:53.571 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:53.572 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.572 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.572 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:53.572 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168173+0000", 2024-04-25T20:09:53.572 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7663999999999997e-05 2024-04-25T20:09:53.572 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.572 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.572 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:53.572 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168182+0000", 2024-04-25T20:09:53.572 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0650000000000005e-06 2024-04-25T20:09:53.572 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.572 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.572 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:53.572 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168188+0000", 2024-04-25T20:09:53.572 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0589999999999999e-06 2024-04-25T20:09:53.573 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.573 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.573 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:53.573 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168239+0000", 2024-04-25T20:09:53.573 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1467e-05 2024-04-25T20:09:53.573 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.573 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.573 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:53.573 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168260+0000", 2024-04-25T20:09:53.573 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0531e-05 2024-04-25T20:09:53.573 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.573 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.573 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:53.573 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173268+0000", 2024-04-25T20:09:53.573 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0050087980000000001 2024-04-25T20:09:53.573 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173291+0000", 2024-04-25T20:09:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.268e-05 2024-04-25T20:09:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:33 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'19, mlcod=16'14)", 2024-04-25T20:09:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:09:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 36.392082592000001, 2024-04-25T20:09:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046697049999999997, 2024-04-25T20:09:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:09:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:09:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168645+0000", 2024-04-25T20:09:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.576 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:53.576 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168651+0000", 2024-04-25T20:09:53.576 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1079999999999998e-06 2024-04-25T20:09:53.576 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.576 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.576 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:53.576 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168660+0000", 2024-04-25T20:09:53.576 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0010000000000007e-06 2024-04-25T20:09:53.576 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.576 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.576 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:53.576 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168666+0000", 2024-04-25T20:09:53.576 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7780000000000004e-06 2024-04-25T20:09:53.576 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.576 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168701+0000", 2024-04-25T20:09:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4490999999999999e-05 2024-04-25T20:09:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168729+0000", 2024-04-25T20:09:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8476999999999999e-05 2024-04-25T20:09:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173294+0000", 2024-04-25T20:09:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0045650730000000002 2024-04-25T20:09:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:53.578 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173315+0000", 2024-04-25T20:09:53.578 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1245000000000001e-05 2024-04-25T20:09:53.578 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.578 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:53.578 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.578 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.578 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.578 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:34 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'20, mlcod=16'14)", 2024-04-25T20:09:53.578 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:09:53.578 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 36.391583662999999, 2024-04-25T20:09:53.578 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041943309999999999, 2024-04-25T20:09:53.578 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:53.578 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:53.578 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:53.578 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.578 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:53.578 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:09:53.579 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.579 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.579 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.579 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:53.579 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:09:53.579 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.579 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.579 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.579 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:53.579 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169144+0000", 2024-04-25T20:09:53.579 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:53.579 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.579 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.579 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:53.579 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169150+0000", 2024-04-25T20:09:53.579 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9150000000000001e-06 2024-04-25T20:09:53.580 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.580 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.580 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:53.580 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169159+0000", 2024-04-25T20:09:53.580 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8899999999999996e-06 2024-04-25T20:09:53.580 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.580 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.580 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:53.580 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169169+0000", 2024-04-25T20:09:53.580 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0441e-05 2024-04-25T20:09:53.580 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.580 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.580 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:53.580 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169206+0000", 2024-04-25T20:09:53.580 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6816000000000001e-05 2024-04-25T20:09:53.580 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.580 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.581 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:53.581 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169234+0000", 2024-04-25T20:09:53.581 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8008999999999999e-05 2024-04-25T20:09:53.581 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.581 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.581 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:53.581 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173317+0000", 2024-04-25T20:09:53.581 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0040830160000000001 2024-04-25T20:09:53.581 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.581 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.581 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:53.581 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173339+0000", 2024-04-25T20:09:53.581 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1897000000000001e-05 2024-04-25T20:09:53.581 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.581 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:53.581 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.581 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.582 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.582 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:35 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'14)", 2024-04-25T20:09:53.582 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:09:53.582 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 36.391091344000003, 2024-04-25T20:09:53.582 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037236529999999999, 2024-04-25T20:09:53.582 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:53.582 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:53.582 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:53.582 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.582 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:53.582 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:09:53.582 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.582 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.582 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.582 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:53.582 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:09:53.583 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.583 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.583 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.583 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:53.583 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169636+0000", 2024-04-25T20:09:53.583 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:53.583 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.583 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.583 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:53.583 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169678+0000", 2024-04-25T20:09:53.583 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1752000000000003e-05 2024-04-25T20:09:53.583 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.583 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.583 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:53.584 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169688+0000", 2024-04-25T20:09:53.584 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0098000000000001e-05 2024-04-25T20:09:53.584 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.584 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.584 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:53.584 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169694+0000", 2024-04-25T20:09:53.584 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2349999999999996e-06 2024-04-25T20:09:53.584 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.584 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.584 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:53.584 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169733+0000", 2024-04-25T20:09:53.584 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8481e-05 2024-04-25T20:09:53.584 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.584 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.584 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:53.585 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169750+0000", 2024-04-25T20:09:53.585 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7509000000000001e-05 2024-04-25T20:09:53.585 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.585 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.585 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:53.585 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173340+0000", 2024-04-25T20:09:53.585 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0035898449999999999 2024-04-25T20:09:53.585 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.585 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.585 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:53.585 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173361+0000", 2024-04-25T20:09:53.585 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.039e-05 2024-04-25T20:09:53.585 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.585 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:53.585 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.586 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.586 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.586 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:36 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'22, mlcod=16'14)", 2024-04-25T20:09:53.586 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:09:53.586 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 36.390591284000003, 2024-04-25T20:09:53.586 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032472120000000002, 2024-04-25T20:09:53.586 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:53.586 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:53.586 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:53.586 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.586 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:53.586 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:09:53.586 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.586 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.586 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.587 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:53.587 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:09:53.587 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.587 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.587 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.587 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:53.587 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:09:53.587 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:53.587 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.587 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.587 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:53.587 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170152+0000", 2024-04-25T20:09:53.587 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5798e-05 2024-04-25T20:09:53.587 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.588 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.588 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:53.588 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170161+0000", 2024-04-25T20:09:53.588 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.064e-06 2024-04-25T20:09:53.588 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.588 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.588 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:53.588 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170167+0000", 2024-04-25T20:09:53.588 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8390000000000003e-06 2024-04-25T20:09:53.588 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.588 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.588 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:53.588 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170202+0000", 2024-04-25T20:09:53.588 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.5120999999999998e-05 2024-04-25T20:09:53.589 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.589 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.589 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:53.589 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170219+0000", 2024-04-25T20:09:53.589 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6929999999999999e-05 2024-04-25T20:09:53.589 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.589 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.589 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:53.589 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173362+0000", 2024-04-25T20:09:53.589 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0031427619999999999 2024-04-25T20:09:53.589 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.589 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.589 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:53.589 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173384+0000", 2024-04-25T20:09:53.590 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2237000000000001e-05 2024-04-25T20:09:53.590 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.590 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:53.590 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.590 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.590 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.590 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:40 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'26, mlcod=16'24)", 2024-04-25T20:09:53.590 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:09:53.590 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 36.383553372000002, 2024-04-25T20:09:53.590 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028727330000000001, 2024-04-25T20:09:53.590 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:53.590 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:53.590 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:53.590 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.591 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:53.591 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:09:53.591 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.591 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.591 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.591 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:53.591 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:09:53.591 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.591 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.591 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.591 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:53.591 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177173+0000", 2024-04-25T20:09:53.591 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999976 2024-04-25T20:09:53.591 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.591 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.592 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:53.592 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177186+0000", 2024-04-25T20:09:53.592 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3533000000000001e-05 2024-04-25T20:09:53.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.592 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.592 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:53.592 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177209+0000", 2024-04-25T20:09:53.592 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3280000000000001e-05 2024-04-25T20:09:53.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.592 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.592 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:53.592 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177220+0000", 2024-04-25T20:09:53.592 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0465000000000001e-05 2024-04-25T20:09:53.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.592 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.593 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:53.593 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177264+0000", 2024-04-25T20:09:53.593 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4413999999999999e-05 2024-04-25T20:09:53.593 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.593 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.593 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:53.593 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177289+0000", 2024-04-25T20:09:53.593 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4783999999999999e-05 2024-04-25T20:09:53.593 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.593 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.593 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:53.593 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.179990+0000", 2024-04-25T20:09:53.593 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027012939999999999 2024-04-25T20:09:53.593 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.593 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.594 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:53.594 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.180048+0000", 2024-04-25T20:09:53.594 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7299000000000001e-05 2024-04-25T20:09:53.594 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.594 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:53.594 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.594 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.594 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.594 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:67 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'41, mlcod=16'39)", 2024-04-25T20:09:53.594 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:09:53.594 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 36.34879188, 2024-04-25T20:09:53.594 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064192549999999996, 2024-04-25T20:09:53.594 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:53.594 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:53.595 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:53.595 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.595 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:53.595 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:09:53.595 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.595 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.595 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.595 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:53.595 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:09:53.595 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.595 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.595 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.595 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:53.595 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211935+0000", 2024-04-25T20:09:53.596 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999981 2024-04-25T20:09:53.596 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.596 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.596 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:53.596 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211948+0000", 2024-04-25T20:09:53.596 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3531e-05 2024-04-25T20:09:53.596 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.596 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.596 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:53.596 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211967+0000", 2024-04-25T20:09:53.596 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8326999999999999e-05 2024-04-25T20:09:53.596 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.596 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.596 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:53.596 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211975+0000", 2024-04-25T20:09:53.597 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8589999999999994e-06 2024-04-25T20:09:53.597 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.597 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.597 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:53.597 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212020+0000", 2024-04-25T20:09:53.597 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4626999999999998e-05 2024-04-25T20:09:53.597 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.597 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.597 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:53.597 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212045+0000", 2024-04-25T20:09:53.597 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4508999999999999e-05 2024-04-25T20:09:53.597 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.597 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.597 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:53.598 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218316+0000", 2024-04-25T20:09:53.598 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0062711640000000001 2024-04-25T20:09:53.598 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.598 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.598 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:53.598 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218356+0000", 2024-04-25T20:09:53.598 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9993000000000001e-05 2024-04-25T20:09:53.598 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.598 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:53.598 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.598 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.598 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.598 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:68 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-04-25T20:09:53.598 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:09:53.598 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 36.348342866000003, 2024-04-25T20:09:53.599 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011231365, 2024-04-25T20:09:53.599 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:53.599 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:53.599 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:53.599 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.599 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:53.599 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:09:53.599 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.599 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.599 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.599 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:53.599 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:09:53.599 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.599 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.600 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.600 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:53.600 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:09:53.600 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:53.600 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.600 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.600 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:53.600 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212392+0000", 2024-04-25T20:09:53.600 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.79e-06 2024-04-25T20:09:53.600 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.600 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.600 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:53.600 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212406+0000", 2024-04-25T20:09:53.600 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3577000000000001e-05 2024-04-25T20:09:53.600 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.600 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.601 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:53.601 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212414+0000", 2024-04-25T20:09:53.601 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6510000000000001e-06 2024-04-25T20:09:53.601 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.601 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.601 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:53.601 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217114+0000", 2024-04-25T20:09:53.601 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047005379999999998 2024-04-25T20:09:53.601 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.601 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.601 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:53.601 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217141+0000", 2024-04-25T20:09:53.601 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6984000000000001e-05 2024-04-25T20:09:53.601 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.601 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.601 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:53.601 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223563+0000", 2024-04-25T20:09:53.602 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064216999999999998 2024-04-25T20:09:53.602 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.602 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.602 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:53.602 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223617+0000", 2024-04-25T20:09:53.602 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4011000000000002e-05 2024-04-25T20:09:53.602 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.602 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:53.602 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.602 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.602 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.602 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:69 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-04-25T20:09:53.602 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:09:53.602 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 36.347810676000002, 2024-04-25T20:09:53.602 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.010730094000000001, 2024-04-25T20:09:53.603 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:53.603 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:53.603 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:53.603 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.603 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:53.603 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:09:53.603 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.603 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.603 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.603 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:53.603 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:09:53.603 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.603 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.603 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.603 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:53.604 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212917+0000", 2024-04-25T20:09:53.604 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:53.604 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.604 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.604 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:53.604 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212957+0000", 2024-04-25T20:09:53.604 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0692000000000003e-05 2024-04-25T20:09:53.604 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.604 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.604 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:53.604 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212973+0000", 2024-04-25T20:09:53.604 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.575e-05 2024-04-25T20:09:53.604 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.604 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.604 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:53.605 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212980+0000", 2024-04-25T20:09:53.605 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1679999999999998e-06 2024-04-25T20:09:53.605 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.605 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.605 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:53.605 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217449+0000", 2024-04-25T20:09:53.605 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.004468367 2024-04-25T20:09:53.605 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.605 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.605 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:53.605 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217470+0000", 2024-04-25T20:09:53.605 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0749000000000001e-05 2024-04-25T20:09:53.605 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.605 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.605 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:53.605 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223621+0000", 2024-04-25T20:09:53.606 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0061512880000000004 2024-04-25T20:09:53.606 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.606 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.606 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:53.606 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223648+0000", 2024-04-25T20:09:53.606 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6961000000000002e-05 2024-04-25T20:09:53.606 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.606 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:53.606 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.606 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.606 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.606 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:70 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'44, mlcod=16'39)", 2024-04-25T20:09:53.606 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:09:53.606 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 36.347051948000001, 2024-04-25T20:09:53.606 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.013553358, 2024-04-25T20:09:53.607 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:53.607 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:53.607 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:53.607 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.607 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:53.607 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:09:53.607 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.607 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.607 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.607 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:53.607 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:09:53.607 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.607 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.607 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.607 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:53.607 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:09:53.608 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:53.608 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.608 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.608 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:53.608 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213682+0000", 2024-04-25T20:09:53.608 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3439999999999997e-06 2024-04-25T20:09:53.608 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.608 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.608 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:53.608 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213693+0000", 2024-04-25T20:09:53.608 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1049999999999999e-05 2024-04-25T20:09:53.608 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.608 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.608 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:53.608 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213699+0000", 2024-04-25T20:09:53.608 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9399999999999999e-06 2024-04-25T20:09:53.609 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.609 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.609 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:53.609 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218086+0000", 2024-04-25T20:09:53.609 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0043871240000000001 2024-04-25T20:09:53.609 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.609 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.609 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:53.609 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218106+0000", 2024-04-25T20:09:53.609 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.997e-05 2024-04-25T20:09:53.609 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.609 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.609 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:53.609 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.227186+0000", 2024-04-25T20:09:53.609 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.009079574 2024-04-25T20:09:53.609 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.609 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.610 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:53.610 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.227230+0000", 2024-04-25T20:09:53.610 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4147999999999998e-05 2024-04-25T20:09:53.610 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.610 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:53.610 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.610 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.610 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.610 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:106 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-04-25T20:09:53.610 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:09:53.610 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 35.975912848, 2024-04-25T20:09:53.610 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002936756, 2024-04-25T20:09:53.610 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:53.610 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:53.610 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:53.611 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.611 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:53.611 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:09:53.611 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.611 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.611 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.611 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:53.611 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:09:53.611 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.611 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.611 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.611 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:53.611 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584815+0000", 2024-04-25T20:09:53.611 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:53.612 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.612 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.612 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:53.612 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584860+0000", 2024-04-25T20:09:53.612 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5161999999999999e-05 2024-04-25T20:09:53.612 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.612 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.612 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:53.612 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584874+0000", 2024-04-25T20:09:53.612 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4464000000000001e-05 2024-04-25T20:09:53.612 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.612 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.612 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:53.612 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584881+0000", 2024-04-25T20:09:53.612 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.409e-06 2024-04-25T20:09:53.612 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.613 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.613 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:53.613 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584915+0000", 2024-04-25T20:09:53.613 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4007000000000002e-05 2024-04-25T20:09:53.613 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.613 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.613 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:53.613 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584933+0000", 2024-04-25T20:09:53.613 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7737999999999999e-05 2024-04-25T20:09:53.613 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.613 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.613 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:53.613 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587717+0000", 2024-04-25T20:09:53.613 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002784711 2024-04-25T20:09:53.613 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.614 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.614 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:53.614 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587752+0000", 2024-04-25T20:09:53.614 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4915e-05 2024-04-25T20:09:53.614 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.614 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:53.614 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.614 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.614 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.614 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:107 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'69, mlcod=16'64)", 2024-04-25T20:09:53.614 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:09:53.614 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 35.975837546000001, 2024-04-25T20:09:53.614 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028906270000000002, 2024-04-25T20:09:53.615 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:53.615 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:53.615 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:53.615 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.615 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:53.615 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:09:53.615 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.615 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.615 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.615 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:53.615 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:09:53.615 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.615 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.615 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.615 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:53.615 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584890+0000", 2024-04-25T20:09:53.615 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:53.616 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.616 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.616 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:53.616 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584895+0000", 2024-04-25T20:09:53.616 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2069999999999997e-06 2024-04-25T20:09:53.616 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.616 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.616 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:53.616 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584906+0000", 2024-04-25T20:09:53.616 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.117e-05 2024-04-25T20:09:53.616 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.616 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.616 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:53.616 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584910+0000", 2024-04-25T20:09:53.616 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4190000000000004e-06 2024-04-25T20:09:53.617 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.617 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.617 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:53.617 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585310+0000", 2024-04-25T20:09:53.617 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00040002700000000001 2024-04-25T20:09:53.617 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.617 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.617 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:53.617 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585329+0000", 2024-04-25T20:09:53.617 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8838000000000002e-05 2024-04-25T20:09:53.617 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.617 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.617 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:53.617 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587754+0000", 2024-04-25T20:09:53.618 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024252689999999999 2024-04-25T20:09:53.618 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.618 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.618 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:53.618 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587781+0000", 2024-04-25T20:09:53.618 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7206999999999999e-05 2024-04-25T20:09:53.618 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.618 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:53.618 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.618 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.618 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.618 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:108 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'70, mlcod=16'64)", 2024-04-25T20:09:53.618 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:09:53.618 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 35.975812394999998, 2024-04-25T20:09:53.618 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028924879999999999, 2024-04-25T20:09:53.619 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:53.619 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:53.619 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:53.619 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.619 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:53.619 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:09:53.619 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.619 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.619 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.619 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:53.619 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:09:53.619 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.619 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.619 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.619 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:53.619 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:09:53.619 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:53.620 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.620 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.620 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:53.620 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584919+0000", 2024-04-25T20:09:53.620 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.557e-06 2024-04-25T20:09:53.620 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.620 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.620 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:53.620 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584925+0000", 2024-04-25T20:09:53.620 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8619999999999999e-06 2024-04-25T20:09:53.620 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.620 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.620 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:53.620 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584931+0000", 2024-04-25T20:09:53.620 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2539999999999998e-06 2024-04-25T20:09:53.620 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.620 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.621 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:53.621 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585550+0000", 2024-04-25T20:09:53.621 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00061821800000000002 2024-04-25T20:09:53.621 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.621 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.621 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:53.621 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585567+0000", 2024-04-25T20:09:53.621 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7499e-05 2024-04-25T20:09:53.621 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.621 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.621 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:53.621 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587783+0000", 2024-04-25T20:09:53.621 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0022156200000000002 2024-04-25T20:09:53.621 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.621 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.621 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:53.622 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587808+0000", 2024-04-25T20:09:53.622 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5729e-05 2024-04-25T20:09:53.622 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.622 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:53.622 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.622 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.622 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.622 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:112 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'74, mlcod=16'66)", 2024-04-25T20:09:53.622 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:09:53.622 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 35.973654773, 2024-04-25T20:09:53.622 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00318724, 2024-04-25T20:09:53.622 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:53.622 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:53.622 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:53.622 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.623 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:53.623 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:09:53.623 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.623 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.623 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.623 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:53.623 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:09:53.623 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.623 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.623 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.623 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:53.623 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587073+0000", 2024-04-25T20:09:53.623 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:53.623 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.623 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.624 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:53.624 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587136+0000", 2024-04-25T20:09:53.624 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2928999999999993e-05 2024-04-25T20:09:53.624 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.624 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.624 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:53.624 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587147+0000", 2024-04-25T20:09:53.624 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1182999999999999e-05 2024-04-25T20:09:53.624 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.624 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.624 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:53.624 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587153+0000", 2024-04-25T20:09:53.624 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9059999999999999e-06 2024-04-25T20:09:53.624 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.624 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.625 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:53.625 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587306+0000", 2024-04-25T20:09:53.625 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000152848 2024-04-25T20:09:53.625 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.625 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.625 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:53.625 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587330+0000", 2024-04-25T20:09:53.625 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4304e-05 2024-04-25T20:09:53.625 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.625 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.625 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:53.625 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.590230+0000", 2024-04-25T20:09:53.625 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029004629999999998 2024-04-25T20:09:53.625 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.625 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.626 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:53.626 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.590261+0000", 2024-04-25T20:09:53.626 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0323000000000001e-05 2024-04-25T20:09:53.626 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.626 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:53.626 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.626 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.626 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.626 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:118 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'80, mlcod=16'78)", 2024-04-25T20:09:53.626 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:09:53.626 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 35.968868194000002, 2024-04-25T20:09:53.626 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029618959999999999, 2024-04-25T20:09:53.626 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:53.627 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:53.627 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:53.627 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.627 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:53.627 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:09:53.627 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.627 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.627 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.627 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:53.627 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:09:53.627 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.627 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.627 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.627 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:53.627 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591859+0000", 2024-04-25T20:09:53.628 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:53.628 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.628 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.628 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:53.628 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591918+0000", 2024-04-25T20:09:53.628 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8842999999999999e-05 2024-04-25T20:09:53.628 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.628 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.628 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:53.628 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591933+0000", 2024-04-25T20:09:53.628 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4600000000000001e-05 2024-04-25T20:09:53.628 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.628 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.628 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:53.629 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591939+0000", 2024-04-25T20:09:53.629 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4559999999999997e-06 2024-04-25T20:09:53.629 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.629 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.629 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:53.629 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591990+0000", 2024-04-25T20:09:53.629 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0318999999999997e-05 2024-04-25T20:09:53.629 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.629 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.629 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:53.629 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.592008+0000", 2024-04-25T20:09:53.629 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8819000000000001e-05 2024-04-25T20:09:53.629 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.629 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.629 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:53.630 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.594784+0000", 2024-04-25T20:09:53.630 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027750850000000001 2024-04-25T20:09:53.630 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.630 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.630 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:53.630 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.594822+0000", 2024-04-25T20:09:53.630 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8522000000000001e-05 2024-04-25T20:09:53.630 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.630 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:53.630 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.630 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.630 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.630 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:139 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'97, mlcod=17'87)", 2024-04-25T20:09:53.630 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:09:53.630 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 35.434831588999998, 2024-04-25T20:09:53.631 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048546290000000001, 2024-04-25T20:09:53.631 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:53.631 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:53.631 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:53.631 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.631 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:53.631 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:09:53.631 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.631 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.631 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.631 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:53.631 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:09:53.631 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.631 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.631 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.632 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:53.632 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125896+0000", 2024-04-25T20:09:53.632 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:53.632 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.632 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.632 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:53.632 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125908+0000", 2024-04-25T20:09:53.632 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.137e-05 2024-04-25T20:09:53.632 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.632 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.632 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:53.632 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125915+0000", 2024-04-25T20:09:53.632 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3939999999999998e-06 2024-04-25T20:09:53.632 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.633 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.633 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:53.633 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125922+0000", 2024-04-25T20:09:53.633 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1930000000000004e-06 2024-04-25T20:09:53.633 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.633 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.633 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:53.633 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125959+0000", 2024-04-25T20:09:53.633 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6701e-05 2024-04-25T20:09:53.633 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.633 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.633 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:53.633 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125972+0000", 2024-04-25T20:09:53.633 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2578000000000001e-05 2024-04-25T20:09:53.633 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.633 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.634 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:53.634 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130719+0000", 2024-04-25T20:09:53.634 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047475800000000004 2024-04-25T20:09:53.634 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.634 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.634 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:53.634 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130751+0000", 2024-04-25T20:09:53.634 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2283000000000002e-05 2024-04-25T20:09:53.634 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.634 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:53.634 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.634 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.634 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.634 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:140 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'98, mlcod=17'87)", 2024-04-25T20:09:53.635 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:09:53.635 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 35.434406238000001, 2024-04-25T20:09:53.635 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048310990000000002, 2024-04-25T20:09:53.635 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:53.635 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:53.635 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:53.635 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.635 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:53.635 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:09:53.635 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.635 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.635 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.635 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:53.635 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:09:53.635 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.636 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.636 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.636 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:53.636 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:09:53.636 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:53.636 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.636 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.636 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:53.636 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126427+0000", 2024-04-25T20:09:53.636 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000105572 2024-04-25T20:09:53.636 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.636 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.636 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:53.636 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126435+0000", 2024-04-25T20:09:53.636 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1920000000000005e-06 2024-04-25T20:09:53.636 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.637 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.637 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:53.637 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126439+0000", 2024-04-25T20:09:53.637 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6950000000000002e-06 2024-04-25T20:09:53.637 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.637 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.637 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:53.637 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126471+0000", 2024-04-25T20:09:53.637 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1699000000000003e-05 2024-04-25T20:09:53.637 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.637 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.637 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:53.637 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126484+0000", 2024-04-25T20:09:53.637 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.343e-05 2024-04-25T20:09:53.637 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.637 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.638 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:53.638 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131131+0000", 2024-04-25T20:09:53.638 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046471480000000003 2024-04-25T20:09:53.638 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.638 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.638 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:53.638 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131153+0000", 2024-04-25T20:09:53.638 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1753e-05 2024-04-25T20:09:53.638 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.638 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:53.638 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.638 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.638 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.638 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:141 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'99, mlcod=17'87)", 2024-04-25T20:09:53.639 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:09:53.639 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 35.433751293999997, 2024-04-25T20:09:53.639 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.004496847, 2024-04-25T20:09:53.639 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:53.639 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:53.639 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:53.639 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.639 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:53.639 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:09:53.639 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.639 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.639 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.639 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:53.639 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:09:53.639 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:53.639 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.640 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.640 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:53.640 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:09:53.640 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:53.640 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.640 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.640 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:53.640 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126980+0000", 2024-04-25T20:09:53.640 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.619e-06 2024-04-25T20:09:53.640 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.640 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.640 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:53.640 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126987+0000", 2024-04-25T20:09:53.640 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7220000000000002e-06 2024-04-25T20:09:53.640 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.640 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.640 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:53.641 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126992+0000", 2024-04-25T20:09:53.641 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8679999999999998e-06 2024-04-25T20:09:53.641 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.641 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.641 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:53.641 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130772+0000", 2024-04-25T20:09:53.641 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037803120000000001 2024-04-25T20:09:53.641 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.641 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.641 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:53.641 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130786+0000", 2024-04-25T20:09:53.641 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3592e-05 2024-04-25T20:09:53.641 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.641 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.641 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:53.641 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131451+0000", 2024-04-25T20:09:53.642 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000665206 2024-04-25T20:09:53.642 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:53.642 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:53.642 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:53.642 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131474+0000", 2024-04-25T20:09:53.642 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2957999999999999e-05 2024-04-25T20:09:53.642 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.642 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:53.642 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.642 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:53.642 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:53.642 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:53.644 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T20:09:53.644 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:53.643+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:53.676 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T20:09:53.744 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:53.744+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:53.793 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:53.793 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:53.793 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:09:53.793 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:53.803 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T20:09:53.809 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:09:53.809 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:09:53.826 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:09:53.827 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:09:53.843 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:09:53.843 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:09:53.845 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:53.845+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:53.861 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:09:53.861 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:09:53.879 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:09:53.879 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:09:53.896 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:09:53.896 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:09:53.914 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:09:53.914 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:09:53.917 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:53.917 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:53.926 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T20:09:53.932 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:09:53.933 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:09:53.946 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:53.946+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:54.038 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:54.038 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:54.046 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T20:09:54.048 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:54.047+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:54.148 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:54.148+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:54.152 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:54.152 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:54.162 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T20:09:54.249 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:54.248+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:54.259 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is set 2024-04-25T20:09:54.310 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:54.310 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:54.319 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T20:09:54.360 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:54.360+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:54.451 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:54.451+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:54.465 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:54.465 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:54.474 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T20:09:54.552 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:54.552+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:54.587 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:54.587 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:54.596 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T20:09:54.653 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:54.652+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:54.698 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:54.698 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:54.706 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T20:09:54.791 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:54.790+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:54.850 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:54.850 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:54.854 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:54.854+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:54.859 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T20:09:54.955 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:54.955+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:55.002 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:55.002 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:55.011 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T20:09:55.056 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:55.055+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:55.113 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:55.113 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:55.123 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T20:09:55.167 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:55.166+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:55.225 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:55.225 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:55.235 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T20:09:55.257 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:55.257+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:55.358 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:55.358+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:55.376 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:55.376 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:55.385 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T20:09:55.459 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:55.458+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:55.486 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:55.487 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:55.495 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T20:09:55.560 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:55.560+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:55.637 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:55.637 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:55.646 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T20:09:55.660 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:55.660+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:55.747 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:55.747 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:55.758 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T20:09:55.761 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:55.760+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:55.859 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:55.860 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:55.862 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:55.862+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:55.868 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T20:09:55.944 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T20:09:55.970 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:55.970 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:55.979 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T20:09:55.988 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:55.988+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:56.064 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:56.063+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:56.097 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:09:56.097 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:09:56.114 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:56.114 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:56.114 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:09:56.115 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:56.115 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:09:56.115 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:09:56.124 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T20:09:56.131 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:09:56.132 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:09:56.149 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:09:56.149 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:09:56.167 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:09:56.167 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:09:56.167 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:56.166+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:56.183 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:09:56.183 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:09:56.200 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:09:56.200 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:09:56.217 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:09:56.218 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:09:56.231 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:56.231 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:56.231 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:09:56.231 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:09:56.232 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:56.237 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6, 7] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-25T20:09:56.237 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T20:09:56.237 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-04-25T20:09:56.238 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-04-25T20:09:56.241 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T20:09:56.273 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-25T20:09:56.281 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:56.281+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:56.369 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:56.368+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:56.390 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:56.391 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:09:56.391 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:09:56.391 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:09:56.391 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:56.400 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T20:09:56.484 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:56.484+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:56.549 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:56.549 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:56.550 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:09:56.550 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:56.558 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T20:09:56.561 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:09:56.561 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:56.561 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 31, 2024-04-25T20:09:56.561 INFO:teuthology.orchestra.run.smithi039.stdout: "fsid": "975ceafd-555d-4f71-81e4-1f70364da00f", 2024-04-25T20:09:56.561 INFO:teuthology.orchestra.run.smithi039.stdout: "created": "2024-04-25T20:09:03.377320+0000", 2024-04-25T20:09:56.561 INFO:teuthology.orchestra.run.smithi039.stdout: "modified": "2024-04-25T20:09:54.224639+0000", 2024-04-25T20:09:56.561 INFO:teuthology.orchestra.run.smithi039.stdout: "last_up_change": "2024-04-25T20:09:13.740341+0000", 2024-04-25T20:09:56.561 INFO:teuthology.orchestra.run.smithi039.stdout: "last_in_change": "2024-04-25T20:09:40.137871+0000", 2024-04-25T20:09:56.561 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T20:09:56.561 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_num": 5806080, 2024-04-25T20:09:56.561 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_set": [ 2024-04-25T20:09:56.561 INFO:teuthology.orchestra.run.smithi039.stdout: "nodeep-scrub", 2024-04-25T20:09:56.562 INFO:teuthology.orchestra.run.smithi039.stdout: "noscrub", 2024-04-25T20:09:56.562 INFO:teuthology.orchestra.run.smithi039.stdout: "pglog_hardlimit", 2024-04-25T20:09:56.562 INFO:teuthology.orchestra.run.smithi039.stdout: "purged_snapdirs", 2024-04-25T20:09:56.562 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_deletes", 2024-04-25T20:09:56.562 INFO:teuthology.orchestra.run.smithi039.stdout: "sortbitwise" 2024-04-25T20:09:56.562 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-04-25T20:09:56.562 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_version": 4, 2024-04-25T20:09:56.562 INFO:teuthology.orchestra.run.smithi039.stdout: "full_ratio": 0.94999998807907104, 2024-04-25T20:09:56.562 INFO:teuthology.orchestra.run.smithi039.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-04-25T20:09:56.562 INFO:teuthology.orchestra.run.smithi039.stdout: "nearfull_ratio": 0.85000002384185791, 2024-04-25T20:09:56.562 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_snapshot": "", 2024-04-25T20:09:56.562 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_max": 2, 2024-04-25T20:09:56.562 INFO:teuthology.orchestra.run.smithi039.stdout: "max_osd": 8, 2024-04-25T20:09:56.562 INFO:teuthology.orchestra.run.smithi039.stdout: "require_min_compat_client": "luminous", 2024-04-25T20:09:56.563 INFO:teuthology.orchestra.run.smithi039.stdout: "min_compat_client": "jewel", 2024-04-25T20:09:56.563 INFO:teuthology.orchestra.run.smithi039.stdout: "require_osd_release": "reef", 2024-04-25T20:09:56.563 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_crimson": false, 2024-04-25T20:09:56.563 INFO:teuthology.orchestra.run.smithi039.stdout: "pools": [ 2024-04-25T20:09:56.563 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.563 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 1, 2024-04-25T20:09:56.563 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": "rbd", 2024-04-25T20:09:56.563 INFO:teuthology.orchestra.run.smithi039.stdout: "create_time": "2024-04-25T20:09:14.202057+0000", 2024-04-25T20:09:56.563 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-04-25T20:09:56.563 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_names": "hashpspool", 2024-04-25T20:09:56.563 INFO:teuthology.orchestra.run.smithi039.stdout: "type": 1, 2024-04-25T20:09:56.563 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 2, 2024-04-25T20:09:56.563 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": 1, 2024-04-25T20:09:56.563 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_rule": 0, 2024-04-25T20:09:56.563 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_count": 0, 2024-04-25T20:09:56.564 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_target": 0, 2024-04-25T20:09:56.564 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T20:09:56.564 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T20:09:56.564 INFO:teuthology.orchestra.run.smithi039.stdout: "object_hash": 2, 2024-04-25T20:09:56.564 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscale_mode": "off", 2024-04-25T20:09:56.564 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": 8, 2024-04-25T20:09:56.564 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num": 8, 2024-04-25T20:09:56.564 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num_target": 8, 2024-04-25T20:09:56.564 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_target": 8, 2024-04-25T20:09:56.564 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_pending": 8, 2024-04-25T20:09:56.564 INFO:teuthology.orchestra.run.smithi039.stdout: "last_pg_merge_meta": { 2024-04-25T20:09:56.564 INFO:teuthology.orchestra.run.smithi039.stdout: "source_pgid": "0.0", 2024-04-25T20:09:56.564 INFO:teuthology.orchestra.run.smithi039.stdout: "ready_epoch": 0, 2024-04-25T20:09:56.564 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_started": 0, 2024-04-25T20:09:56.565 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_clean": 0, 2024-04-25T20:09:56.565 INFO:teuthology.orchestra.run.smithi039.stdout: "source_version": "0'0", 2024-04-25T20:09:56.565 INFO:teuthology.orchestra.run.smithi039.stdout: "target_version": "0'0" 2024-04-25T20:09:56.565 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.565 INFO:teuthology.orchestra.run.smithi039.stdout: "last_change": "16", 2024-04-25T20:09:56.565 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend": "0", 2024-04-25T20:09:56.565 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T20:09:56.565 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T20:09:56.565 INFO:teuthology.orchestra.run.smithi039.stdout: "auid": 0, 2024-04-25T20:09:56.565 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_mode": "selfmanaged", 2024-04-25T20:09:56.565 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_seq": 0, 2024-04-25T20:09:56.565 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_epoch": 0, 2024-04-25T20:09:56.565 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_snaps": [], 2024-04-25T20:09:56.565 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps": "[]", 2024-04-25T20:09:56.566 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_bytes": 0, 2024-04-25T20:09:56.566 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_objects": 0, 2024-04-25T20:09:56.566 INFO:teuthology.orchestra.run.smithi039.stdout: "tiers": [], 2024-04-25T20:09:56.566 INFO:teuthology.orchestra.run.smithi039.stdout: "tier_of": -1, 2024-04-25T20:09:56.566 INFO:teuthology.orchestra.run.smithi039.stdout: "read_tier": -1, 2024-04-25T20:09:56.566 INFO:teuthology.orchestra.run.smithi039.stdout: "write_tier": -1, 2024-04-25T20:09:56.566 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_mode": "none", 2024-04-25T20:09:56.566 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_bytes": 0, 2024-04-25T20:09:56.566 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_objects": 0, 2024-04-25T20:09:56.566 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T20:09:56.566 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T20:09:56.566 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T20:09:56.566 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_flush_age": 0, 2024-04-25T20:09:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_evict_age": 0, 2024-04-25T20:09:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profile": "", 2024-04-25T20:09:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_params": { 2024-04-25T20:09:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "none" 2024-04-25T20:09:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_period": 0, 2024-04-25T20:09:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_count": 0, 2024-04-25T20:09:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: "use_gmt_hitset": true, 2024-04-25T20:09:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: "min_read_recency_for_promote": 0, 2024-04-25T20:09:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: "min_write_recency_for_promote": 0, 2024-04-25T20:09:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T20:09:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_search_last_n": 0, 2024-04-25T20:09:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: "grade_table": [], 2024-04-25T20:09:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: "stripe_width": 0, 2024-04-25T20:09:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: "expected_num_objects": 0, 2024-04-25T20:09:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: "fast_read": false, 2024-04-25T20:09:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: "options": {}, 2024-04-25T20:09:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: "application_metadata": { 2024-04-25T20:09:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd": {} 2024-04-25T20:09:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: "read_balance": { 2024-04-25T20:09:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: "score_acting": 2.630000114440918, 2024-04-25T20:09:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: "score_stable": 2.630000114440918, 2024-04-25T20:09:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: "optimal_score": 1, 2024-04-25T20:09:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_acting": 2.630000114440918, 2024-04-25T20:09:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_stable": 2.630000114440918, 2024-04-25T20:09:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity_weighted": 0.82155418395996094, 2024-04-25T20:09:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity": 0.93999999761581421, 2024-04-25T20:09:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity_weighted": 0.81999999284744263 2024-04-25T20:09:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-04-25T20:09:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": ".mgr", 2024-04-25T20:09:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: "create_time": "2024-04-25T20:09:15.645688+0000", 2024-04-25T20:09:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-04-25T20:09:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_names": "hashpspool", 2024-04-25T20:09:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: "type": 1, 2024-04-25T20:09:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 2, 2024-04-25T20:09:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": 1, 2024-04-25T20:09:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_rule": 0, 2024-04-25T20:09:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_count": 0, 2024-04-25T20:09:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_target": 0, 2024-04-25T20:09:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T20:09:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T20:09:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: "object_hash": 2, 2024-04-25T20:09:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscale_mode": "off", 2024-04-25T20:09:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": 1, 2024-04-25T20:09:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num": 1, 2024-04-25T20:09:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num_target": 1, 2024-04-25T20:09:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_target": 1, 2024-04-25T20:09:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_pending": 1, 2024-04-25T20:09:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: "last_pg_merge_meta": { 2024-04-25T20:09:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: "source_pgid": "0.0", 2024-04-25T20:09:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: "ready_epoch": 0, 2024-04-25T20:09:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_started": 0, 2024-04-25T20:09:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_clean": 0, 2024-04-25T20:09:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: "source_version": "0'0", 2024-04-25T20:09:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: "target_version": "0'0" 2024-04-25T20:09:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: "last_change": "17", 2024-04-25T20:09:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend": "0", 2024-04-25T20:09:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T20:09:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T20:09:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: "auid": 0, 2024-04-25T20:09:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_mode": "selfmanaged", 2024-04-25T20:09:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_seq": 0, 2024-04-25T20:09:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_epoch": 0, 2024-04-25T20:09:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_snaps": [], 2024-04-25T20:09:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps": "[]", 2024-04-25T20:09:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_bytes": 0, 2024-04-25T20:09:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_objects": 0, 2024-04-25T20:09:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: "tiers": [], 2024-04-25T20:09:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: "tier_of": -1, 2024-04-25T20:09:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: "read_tier": -1, 2024-04-25T20:09:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: "write_tier": -1, 2024-04-25T20:09:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_mode": "none", 2024-04-25T20:09:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_bytes": 0, 2024-04-25T20:09:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_objects": 0, 2024-04-25T20:09:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T20:09:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T20:09:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T20:09:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_flush_age": 0, 2024-04-25T20:09:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_evict_age": 0, 2024-04-25T20:09:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profile": "", 2024-04-25T20:09:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_params": { 2024-04-25T20:09:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "none" 2024-04-25T20:09:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_period": 0, 2024-04-25T20:09:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_count": 0, 2024-04-25T20:09:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: "use_gmt_hitset": true, 2024-04-25T20:09:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: "min_read_recency_for_promote": 0, 2024-04-25T20:09:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: "min_write_recency_for_promote": 0, 2024-04-25T20:09:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T20:09:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_search_last_n": 0, 2024-04-25T20:09:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: "grade_table": [], 2024-04-25T20:09:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: "stripe_width": 0, 2024-04-25T20:09:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: "expected_num_objects": 0, 2024-04-25T20:09:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: "fast_read": false, 2024-04-25T20:09:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: "options": { 2024-04-25T20:09:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_max": 32, 2024-04-25T20:09:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_min": 1 2024-04-25T20:09:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: "application_metadata": { 2024-04-25T20:09:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": {} 2024-04-25T20:09:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: "read_balance": { 2024-04-25T20:09:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: "score_acting": 6.9000000953674316, 2024-04-25T20:09:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: "score_stable": 6.9000000953674316, 2024-04-25T20:09:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: "optimal_score": 0.28999999165534973, 2024-04-25T20:09:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_acting": 2, 2024-04-25T20:09:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_stable": 2, 2024-04-25T20:09:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity_weighted": 0.82155418395996094, 2024-04-25T20:09:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity": 0.93999999761581421, 2024-04-25T20:09:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity_weighted": 0.81999999284744263 2024-04-25T20:09:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-04-25T20:09:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: "osds": [ 2024-04-25T20:09:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 0, 2024-04-25T20:09:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "7a7118ba-4aee-404f-97e3-94c50e2996c4", 2024-04-25T20:09:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-04-25T20:09:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-04-25T20:09:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-04-25T20:09:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-04-25T20:09:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-04-25T20:09:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-04-25T20:09:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 13, 2024-04-25T20:09:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 21, 2024-04-25T20:09:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-04-25T20:09:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-04-25T20:09:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-04-25T20:09:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6816", 2024-04-25T20:09:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31832 2024-04-25T20:09:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6818", 2024-04-25T20:09:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31832 2024-04-25T20:09:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-04-25T20:09:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6820", 2024-04-25T20:09:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31832 2024-04-25T20:09:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6822", 2024-04-25T20:09:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31832 2024-04-25T20:09:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-04-25T20:09:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6828", 2024-04-25T20:09:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31832 2024-04-25T20:09:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6830", 2024-04-25T20:09:56.579 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31832 2024-04-25T20:09:56.579 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.579 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.579 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.579 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-04-25T20:09:56.579 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.579 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.579 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.579 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6824", 2024-04-25T20:09:56.579 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31832 2024-04-25T20:09:56.579 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.579 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.579 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.579 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6826", 2024-04-25T20:09:56.579 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31832 2024-04-25T20:09:56.579 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.580 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.580 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.580 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6818/31832", 2024-04-25T20:09:56.580 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6822/31832", 2024-04-25T20:09:56.580 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6830/31832", 2024-04-25T20:09:56.580 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6826/31832", 2024-04-25T20:09:56.580 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-04-25T20:09:56.580 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-04-25T20:09:56.580 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-04-25T20:09:56.580 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.580 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.580 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.580 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 1, 2024-04-25T20:09:56.580 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "13637dbe-e37f-457f-b68b-8eed1b970617", 2024-04-25T20:09:56.580 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-04-25T20:09:56.581 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-04-25T20:09:56.581 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-04-25T20:09:56.581 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-04-25T20:09:56.581 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-04-25T20:09:56.581 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-04-25T20:09:56.581 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 13, 2024-04-25T20:09:56.581 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 14, 2024-04-25T20:09:56.581 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-04-25T20:09:56.581 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-04-25T20:09:56.581 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-04-25T20:09:56.581 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.581 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.581 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.581 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6808", 2024-04-25T20:09:56.581 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31831 2024-04-25T20:09:56.581 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.582 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.582 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.582 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6809", 2024-04-25T20:09:56.582 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31831 2024-04-25T20:09:56.582 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.582 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.582 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.582 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-04-25T20:09:56.582 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.582 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.582 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.582 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6810", 2024-04-25T20:09:56.582 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31831 2024-04-25T20:09:56.582 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.583 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.583 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.583 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6811", 2024-04-25T20:09:56.583 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31831 2024-04-25T20:09:56.583 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.583 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.583 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.583 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-04-25T20:09:56.583 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.583 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.583 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.583 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6814", 2024-04-25T20:09:56.583 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31831 2024-04-25T20:09:56.583 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.583 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.584 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.584 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6815", 2024-04-25T20:09:56.584 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31831 2024-04-25T20:09:56.584 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.584 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.584 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.584 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-04-25T20:09:56.584 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.584 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.584 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.584 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6812", 2024-04-25T20:09:56.584 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31831 2024-04-25T20:09:56.584 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.584 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.584 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.585 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6813", 2024-04-25T20:09:56.585 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31831 2024-04-25T20:09:56.585 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.585 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.585 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.585 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6809/31831", 2024-04-25T20:09:56.585 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6811/31831", 2024-04-25T20:09:56.585 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6815/31831", 2024-04-25T20:09:56.585 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6813/31831", 2024-04-25T20:09:56.585 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-04-25T20:09:56.585 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-04-25T20:09:56.585 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-04-25T20:09:56.585 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.585 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.585 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.586 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 2, 2024-04-25T20:09:56.586 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "29e20ce2-4776-455d-8cf4-d539e738a6a7", 2024-04-25T20:09:56.586 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-04-25T20:09:56.586 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-04-25T20:09:56.586 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-04-25T20:09:56.586 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-04-25T20:09:56.586 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-04-25T20:09:56.586 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-04-25T20:09:56.586 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 13, 2024-04-25T20:09:56.586 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 0, 2024-04-25T20:09:56.586 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-04-25T20:09:56.586 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-04-25T20:09:56.586 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-04-25T20:09:56.587 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.587 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.587 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.587 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6800", 2024-04-25T20:09:56.587 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31829 2024-04-25T20:09:56.587 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.587 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.587 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.587 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6801", 2024-04-25T20:09:56.587 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31829 2024-04-25T20:09:56.587 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.587 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.587 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.587 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-04-25T20:09:56.587 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.588 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.588 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.588 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6802", 2024-04-25T20:09:56.588 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31829 2024-04-25T20:09:56.588 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.588 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.588 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.588 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6803", 2024-04-25T20:09:56.588 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31829 2024-04-25T20:09:56.588 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.588 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.588 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.588 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-04-25T20:09:56.588 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.588 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.589 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.589 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6806", 2024-04-25T20:09:56.589 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31829 2024-04-25T20:09:56.589 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.589 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.589 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.589 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6807", 2024-04-25T20:09:56.589 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31829 2024-04-25T20:09:56.589 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.589 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.589 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.589 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-04-25T20:09:56.589 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.590 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.590 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.590 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6804", 2024-04-25T20:09:56.590 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31829 2024-04-25T20:09:56.590 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.590 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.590 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.590 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6805", 2024-04-25T20:09:56.590 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31829 2024-04-25T20:09:56.590 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.590 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.590 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.590 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6801/31829", 2024-04-25T20:09:56.590 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6803/31829", 2024-04-25T20:09:56.590 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6807/31829", 2024-04-25T20:09:56.591 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6805/31829", 2024-04-25T20:09:56.591 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-04-25T20:09:56.591 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-04-25T20:09:56.591 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-04-25T20:09:56.591 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.591 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.591 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.591 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 3, 2024-04-25T20:09:56.591 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "a8b34e02-0898-4303-9fd5-62cbba9fec5c", 2024-04-25T20:09:56.591 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-04-25T20:09:56.591 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 0, 2024-04-25T20:09:56.591 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 0, 2024-04-25T20:09:56.591 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-04-25T20:09:56.591 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-04-25T20:09:56.592 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-04-25T20:09:56.592 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 13, 2024-04-25T20:09:56.592 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 14, 2024-04-25T20:09:56.592 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-04-25T20:09:56.592 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-04-25T20:09:56.592 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-04-25T20:09:56.592 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.592 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.592 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.592 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6817", 2024-04-25T20:09:56.592 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31830 2024-04-25T20:09:56.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.592 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.592 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.593 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6819", 2024-04-25T20:09:56.593 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31830 2024-04-25T20:09:56.593 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.593 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.593 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.593 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-04-25T20:09:56.593 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.593 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.593 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.593 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6821", 2024-04-25T20:09:56.593 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31830 2024-04-25T20:09:56.593 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.593 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.594 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.594 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6823", 2024-04-25T20:09:56.594 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31830 2024-04-25T20:09:56.594 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.594 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.594 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.594 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-04-25T20:09:56.594 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.594 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.594 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.594 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6829", 2024-04-25T20:09:56.594 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31830 2024-04-25T20:09:56.594 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.594 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.594 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.595 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6831", 2024-04-25T20:09:56.595 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31830 2024-04-25T20:09:56.595 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.595 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.595 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.595 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-04-25T20:09:56.595 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.595 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.595 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.595 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6825", 2024-04-25T20:09:56.595 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31830 2024-04-25T20:09:56.595 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.595 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.596 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.596 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6827", 2024-04-25T20:09:56.596 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31830 2024-04-25T20:09:56.596 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.596 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.596 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.596 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6819/31830", 2024-04-25T20:09:56.596 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6823/31830", 2024-04-25T20:09:56.596 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6831/31830", 2024-04-25T20:09:56.596 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6827/31830", 2024-04-25T20:09:56.596 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-04-25T20:09:56.596 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-04-25T20:09:56.596 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-04-25T20:09:56.596 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.597 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.597 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.597 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 4, 2024-04-25T20:09:56.597 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "e92b5c67-6908-4fd4-9940-5cb98b5ee847", 2024-04-25T20:09:56.597 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-04-25T20:09:56.597 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-04-25T20:09:56.597 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-04-25T20:09:56.597 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 0.5724334716796875, 2024-04-25T20:09:56.597 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-04-25T20:09:56.597 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-04-25T20:09:56.597 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 13, 2024-04-25T20:09:56.597 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 14, 2024-04-25T20:09:56.597 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-04-25T20:09:56.597 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-04-25T20:09:56.598 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-04-25T20:09:56.598 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.598 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.598 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.598 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6800", 2024-04-25T20:09:56.598 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31724 2024-04-25T20:09:56.598 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.598 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.598 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.598 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6801", 2024-04-25T20:09:56.598 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31724 2024-04-25T20:09:56.598 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.598 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.598 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.599 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-04-25T20:09:56.599 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.599 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.599 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.599 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6802", 2024-04-25T20:09:56.599 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31724 2024-04-25T20:09:56.599 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.599 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.599 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.599 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6803", 2024-04-25T20:09:56.599 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31724 2024-04-25T20:09:56.599 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.599 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.599 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.599 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-04-25T20:09:56.600 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.600 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.600 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.600 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6806", 2024-04-25T20:09:56.600 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31724 2024-04-25T20:09:56.600 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.600 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.600 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.600 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6807", 2024-04-25T20:09:56.600 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31724 2024-04-25T20:09:56.600 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.600 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.600 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.600 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-04-25T20:09:56.600 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.601 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.601 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.601 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6804", 2024-04-25T20:09:56.601 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31724 2024-04-25T20:09:56.601 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.601 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.601 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.601 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6805", 2024-04-25T20:09:56.601 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31724 2024-04-25T20:09:56.601 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.601 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.601 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.601 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.135:6801/31724", 2024-04-25T20:09:56.601 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.135:6803/31724", 2024-04-25T20:09:56.602 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.135:6807/31724", 2024-04-25T20:09:56.602 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.135:6805/31724", 2024-04-25T20:09:56.602 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-04-25T20:09:56.602 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-04-25T20:09:56.602 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-04-25T20:09:56.602 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.602 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.602 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.602 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 5, 2024-04-25T20:09:56.602 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "d74d8fef-0211-413b-bff2-f97776fc1faa", 2024-04-25T20:09:56.602 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-04-25T20:09:56.602 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-04-25T20:09:56.602 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-04-25T20:09:56.602 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 0, 2024-04-25T20:09:56.603 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-04-25T20:09:56.603 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-04-25T20:09:56.603 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 13, 2024-04-25T20:09:56.603 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 14, 2024-04-25T20:09:56.603 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-04-25T20:09:56.603 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-04-25T20:09:56.603 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-04-25T20:09:56.603 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.603 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.603 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.603 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6820", 2024-04-25T20:09:56.603 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31734 2024-04-25T20:09:56.603 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.603 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.603 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.604 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6822", 2024-04-25T20:09:56.604 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31734 2024-04-25T20:09:56.604 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.604 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.604 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.604 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-04-25T20:09:56.604 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.604 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.604 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.604 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6824", 2024-04-25T20:09:56.604 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31734 2024-04-25T20:09:56.604 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.604 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.604 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.604 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6826", 2024-04-25T20:09:56.604 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31734 2024-04-25T20:09:56.605 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.605 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.605 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.605 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-04-25T20:09:56.605 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.605 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.605 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.605 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6830", 2024-04-25T20:09:56.605 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31734 2024-04-25T20:09:56.605 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.605 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.605 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.605 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6831", 2024-04-25T20:09:56.605 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31734 2024-04-25T20:09:56.606 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.606 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.606 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.606 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-04-25T20:09:56.606 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.606 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.606 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.606 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6828", 2024-04-25T20:09:56.606 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31734 2024-04-25T20:09:56.606 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.606 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.606 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.606 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6829", 2024-04-25T20:09:56.606 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31734 2024-04-25T20:09:56.607 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.607 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.607 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.607 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.135:6822/31734", 2024-04-25T20:09:56.607 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.135:6826/31734", 2024-04-25T20:09:56.607 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.135:6831/31734", 2024-04-25T20:09:56.607 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.135:6829/31734", 2024-04-25T20:09:56.607 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-04-25T20:09:56.607 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-04-25T20:09:56.607 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-04-25T20:09:56.607 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.607 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.607 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.607 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 6, 2024-04-25T20:09:56.607 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "7dc84c6e-76d8-4f22-8b17-51f62765762d", 2024-04-25T20:09:56.608 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-04-25T20:09:56.608 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-04-25T20:09:56.608 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-04-25T20:09:56.608 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-04-25T20:09:56.608 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-04-25T20:09:56.608 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-04-25T20:09:56.608 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 13, 2024-04-25T20:09:56.608 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 25, 2024-04-25T20:09:56.608 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-04-25T20:09:56.608 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-04-25T20:09:56.608 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-04-25T20:09:56.608 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.608 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.608 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.609 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6808", 2024-04-25T20:09:56.609 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31725 2024-04-25T20:09:56.609 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.609 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.609 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.609 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6809", 2024-04-25T20:09:56.609 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31725 2024-04-25T20:09:56.609 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.609 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.609 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.609 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-04-25T20:09:56.609 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.609 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.609 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.609 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6810", 2024-04-25T20:09:56.610 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31725 2024-04-25T20:09:56.610 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.610 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.610 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.610 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6811", 2024-04-25T20:09:56.610 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31725 2024-04-25T20:09:56.610 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.610 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.610 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.610 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-04-25T20:09:56.610 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.610 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.610 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.610 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6814", 2024-04-25T20:09:56.611 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31725 2024-04-25T20:09:56.611 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.611 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.611 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.611 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6815", 2024-04-25T20:09:56.611 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31725 2024-04-25T20:09:56.611 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.611 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.611 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.611 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-04-25T20:09:56.611 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.611 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.611 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.611 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6812", 2024-04-25T20:09:56.612 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31725 2024-04-25T20:09:56.612 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.612 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.612 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.612 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6813", 2024-04-25T20:09:56.612 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31725 2024-04-25T20:09:56.612 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.612 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.612 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.612 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.135:6809/31725", 2024-04-25T20:09:56.612 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.135:6811/31725", 2024-04-25T20:09:56.612 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.135:6815/31725", 2024-04-25T20:09:56.612 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.135:6813/31725", 2024-04-25T20:09:56.612 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-04-25T20:09:56.612 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-04-25T20:09:56.612 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-04-25T20:09:56.613 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.613 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.613 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.613 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 7, 2024-04-25T20:09:56.613 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "7adb3187-aed3-48af-868c-7230e9bfc85e", 2024-04-25T20:09:56.613 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-04-25T20:09:56.613 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-04-25T20:09:56.613 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-04-25T20:09:56.613 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-04-25T20:09:56.613 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-04-25T20:09:56.613 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-04-25T20:09:56.613 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 13, 2024-04-25T20:09:56.613 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 25, 2024-04-25T20:09:56.613 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-04-25T20:09:56.613 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-04-25T20:09:56.614 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-04-25T20:09:56.614 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.614 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.614 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.614 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6816", 2024-04-25T20:09:56.614 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31737 2024-04-25T20:09:56.614 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.614 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.614 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.614 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6817", 2024-04-25T20:09:56.614 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31737 2024-04-25T20:09:56.614 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.614 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.615 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.615 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-04-25T20:09:56.615 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.615 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.615 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.615 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6818", 2024-04-25T20:09:56.615 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31737 2024-04-25T20:09:56.615 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.615 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.615 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.615 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6819", 2024-04-25T20:09:56.615 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31737 2024-04-25T20:09:56.615 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.615 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.616 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.616 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-04-25T20:09:56.616 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.616 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.616 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.616 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6825", 2024-04-25T20:09:56.616 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31737 2024-04-25T20:09:56.616 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.616 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.616 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.616 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6827", 2024-04-25T20:09:56.616 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31737 2024-04-25T20:09:56.616 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.616 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.616 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.617 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-04-25T20:09:56.617 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:09:56.617 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.617 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:09:56.617 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6821", 2024-04-25T20:09:56.617 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31737 2024-04-25T20:09:56.617 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.617 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.617 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:09:56.617 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6823", 2024-04-25T20:09:56.617 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31737 2024-04-25T20:09:56.617 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.617 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.617 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.618 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.135:6817/31737", 2024-04-25T20:09:56.618 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.135:6819/31737", 2024-04-25T20:09:56.618 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.135:6827/31737", 2024-04-25T20:09:56.618 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.135:6823/31737", 2024-04-25T20:09:56.618 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-04-25T20:09:56.618 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-04-25T20:09:56.618 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-04-25T20:09:56.618 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.618 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.618 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-04-25T20:09:56.618 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_xinfo": [ 2024-04-25T20:09:56.618 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.618 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 0, 2024-04-25T20:09:56.618 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-04-25T20:09:56.618 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-04-25T20:09:56.619 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-04-25T20:09:56.619 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540138322906710015, 2024-04-25T20:09:56.619 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-04-25T20:09:56.619 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-04-25T20:09:10.683465+0000", 2024-04-25T20:09:56.619 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-04-25T20:09:56.619 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.619 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.619 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 1, 2024-04-25T20:09:56.619 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-04-25T20:09:56.619 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-04-25T20:09:56.619 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-04-25T20:09:56.619 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540138322906710015, 2024-04-25T20:09:56.619 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-04-25T20:09:56.619 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-04-25T20:09:10.722423+0000", 2024-04-25T20:09:56.619 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-04-25T20:09:56.619 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.620 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.620 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 2, 2024-04-25T20:09:56.620 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-04-25T20:09:56.620 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-04-25T20:09:56.620 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-04-25T20:09:56.620 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540138322906710015, 2024-04-25T20:09:56.620 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-04-25T20:09:56.620 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-04-25T20:09:10.712691+0000", 2024-04-25T20:09:56.620 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-04-25T20:09:56.620 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.620 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.620 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 3, 2024-04-25T20:09:56.620 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-04-25T20:09:56.620 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-04-25T20:09:56.621 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-04-25T20:09:56.621 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540138322906710015, 2024-04-25T20:09:56.621 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 65536, 2024-04-25T20:09:56.621 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-04-25T20:09:10.673892+0000", 2024-04-25T20:09:56.621 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-04-25T20:09:56.621 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.621 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.621 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 4, 2024-04-25T20:09:56.621 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-04-25T20:09:56.621 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-04-25T20:09:56.621 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-04-25T20:09:56.621 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540138322906710015, 2024-04-25T20:09:56.621 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-04-25T20:09:56.621 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-04-25T20:09:10.768469+0000", 2024-04-25T20:09:56.621 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-04-25T20:09:56.622 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.622 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.622 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 5, 2024-04-25T20:09:56.622 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-04-25T20:09:56.622 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-04-25T20:09:56.622 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-04-25T20:09:56.622 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540138322906710015, 2024-04-25T20:09:56.622 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-04-25T20:09:56.622 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-04-25T20:09:10.783897+0000", 2024-04-25T20:09:56.622 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-04-25T20:09:56.622 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.622 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.622 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 6, 2024-04-25T20:09:56.622 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-04-25T20:09:56.623 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-04-25T20:09:56.623 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-04-25T20:09:56.623 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540138322906710015, 2024-04-25T20:09:56.623 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-04-25T20:09:56.623 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-04-25T20:09:10.686833+0000", 2024-04-25T20:09:56.623 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-04-25T20:09:56.623 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.623 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.623 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 7, 2024-04-25T20:09:56.623 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-04-25T20:09:56.623 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-04-25T20:09:56.623 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-04-25T20:09:56.623 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540138322906710015, 2024-04-25T20:09:56.623 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-04-25T20:09:56.623 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-04-25T20:09:10.739675+0000", 2024-04-25T20:09:56.624 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-04-25T20:09:56.624 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.624 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-04-25T20:09:56.624 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_upmap": [], 2024-04-25T20:09:56.624 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_upmap_items": [], 2024-04-25T20:09:56.624 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_upmap_primaries": [], 2024-04-25T20:09:56.624 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_temp": [], 2024-04-25T20:09:56.624 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_temp": [], 2024-04-25T20:09:56.624 INFO:teuthology.orchestra.run.smithi039.stdout: "blocklist": {}, 2024-04-25T20:09:56.624 INFO:teuthology.orchestra.run.smithi039.stdout: "range_blocklist": {}, 2024-04-25T20:09:56.624 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profiles": { 2024-04-25T20:09:56.624 INFO:teuthology.orchestra.run.smithi039.stdout: "default": { 2024-04-25T20:09:56.624 INFO:teuthology.orchestra.run.smithi039.stdout: "crush-failure-domain": "osd", 2024-04-25T20:09:56.624 INFO:teuthology.orchestra.run.smithi039.stdout: "k": "2", 2024-04-25T20:09:56.624 INFO:teuthology.orchestra.run.smithi039.stdout: "m": "1", 2024-04-25T20:09:56.624 INFO:teuthology.orchestra.run.smithi039.stdout: "plugin": "jerasure", 2024-04-25T20:09:56.625 INFO:teuthology.orchestra.run.smithi039.stdout: "technique": "reed_sol_van" 2024-04-25T20:09:56.625 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.625 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.625 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps_queue": [], 2024-04-25T20:09:56.625 INFO:teuthology.orchestra.run.smithi039.stdout: "new_removed_snaps": [], 2024-04-25T20:09:56.625 INFO:teuthology.orchestra.run.smithi039.stdout: "new_purged_snaps": [], 2024-04-25T20:09:56.625 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_node_flags": {}, 2024-04-25T20:09:56.625 INFO:teuthology.orchestra.run.smithi039.stdout: "device_class_flags": {}, 2024-04-25T20:09:56.625 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_mode": { 2024-04-25T20:09:56.625 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_mode_enabled": false, 2024-04-25T20:09:56.625 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_bucket_count": 0, 2024-04-25T20:09:56.625 INFO:teuthology.orchestra.run.smithi039.stdout: "degraded_stretch_mode": 0, 2024-04-25T20:09:56.625 INFO:teuthology.orchestra.run.smithi039.stdout: "recovering_stretch_mode": 0, 2024-04-25T20:09:56.625 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_mode_bucket": 0 2024-04-25T20:09:56.626 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.626 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:56.628 INFO:tasks.thrashosds.thrasher:j is {'epoch': 31, 'fsid': '975ceafd-555d-4f71-81e4-1f70364da00f', 'created': '2024-04-25T20:09:03.377320+0000', 'modified': '2024-04-25T20:09:54.224639+0000', 'last_up_change': '2024-04-25T20:09:13.740341+0000', 'last_in_change': '2024-04-25T20:09:40.137871+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-04-25T20:09:14.202057+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.630000114440918, 'score_stable': 2.630000114440918, 'optimal_score': 1, 'raw_score_acting': 2.630000114440918, 'raw_score_stable': 2.630000114440918, 'primary_affinity_weighted': 0.8215541839599609, 'average_primary_affinity': 0.9399999976158142, 'average_primary_affinity_weighted': 0.8199999928474426}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-04-25T20:09:15.645688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8215541839599609, 'average_primary_affinity': 0.9399999976158142, 'average_primary_affinity_weighted': 0.8199999928474426}}], 'osds': [{'osd': 0, 'uuid': '7a7118ba-4aee-404f-97e3-94c50e2996c4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6816', 'nonce': 31832}, {'type': 'v1', 'addr': '172.21.15.39:6818', 'nonce': 31832}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6820', 'nonce': 31832}, {'type': 'v1', 'addr': '172.21.15.39:6822', 'nonce': 31832}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6828', 'nonce': 31832}, {'type': 'v1', 'addr': '172.21.15.39:6830', 'nonce': 31832}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6824', 'nonce': 31832}, {'type': 'v1', 'addr': '172.21.15.39:6826', 'nonce': 31832}]}, 'public_addr': '172.21.15.39:6818/31832', 'cluster_addr': '172.21.15.39:6822/31832', 'heartbeat_back_addr': '172.21.15.39:6830/31832', 'heartbeat_front_addr': '172.21.15.39:6826/31832', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '13637dbe-e37f-457f-b68b-8eed1b970617', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 14, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6808', 'nonce': 31831}, {'type': 'v1', 'addr': '172.21.15.39:6809', 'nonce': 31831}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6810', 'nonce': 31831}, {'type': 'v1', 'addr': '172.21.15.39:6811', 'nonce': 31831}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6814', 'nonce': 31831}, {'type': 'v1', 'addr': '172.21.15.39:6815', 'nonce': 31831}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6812', 'nonce': 31831}, {'type': 'v1', 'addr': '172.21.15.39:6813', 'nonce': 31831}]}, 'public_addr': '172.21.15.39:6809/31831', 'cluster_addr': '172.21.15.39:6811/31831', 'heartbeat_back_addr': '172.21.15.39:6815/31831', 'heartbeat_front_addr': '172.21.15.39:6813/31831', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '29e20ce2-4776-455d-8cf4-d539e738a6a7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6800', 'nonce': 31829}, {'type': 'v1', 'addr': '172.21.15.39:6801', 'nonce': 31829}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6802', 'nonce': 31829}, {'type': 'v1', 'addr': '172.21.15.39:6803', 'nonce': 31829}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6806', 'nonce': 31829}, {'type': 'v1', 'addr': '172.21.15.39:6807', 'nonce': 31829}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6804', 'nonce': 31829}, {'type': 'v1', 'addr': '172.21.15.39:6805', 'nonce': 31829}]}, 'public_addr': '172.21.15.39:6801/31829', 'cluster_addr': '172.21.15.39:6803/31829', 'heartbeat_back_addr': '172.21.15.39:6807/31829', 'heartbeat_front_addr': '172.21.15.39:6805/31829', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'a8b34e02-0898-4303-9fd5-62cbba9fec5c', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 14, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6817', 'nonce': 31830}, {'type': 'v1', 'addr': '172.21.15.39:6819', 'nonce': 31830}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6821', 'nonce': 31830}, {'type': 'v1', 'addr': '172.21.15.39:6823', 'nonce': 31830}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6829', 'nonce': 31830}, {'type': 'v1', 'addr': '172.21.15.39:6831', 'nonce': 31830}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6825', 'nonce': 31830}, {'type': 'v1', 'addr': '172.21.15.39:6827', 'nonce': 31830}]}, 'public_addr': '172.21.15.39:6819/31830', 'cluster_addr': '172.21.15.39:6823/31830', 'heartbeat_back_addr': '172.21.15.39:6831/31830', 'heartbeat_front_addr': '172.21.15.39:6827/31830', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'e92b5c67-6908-4fd4-9940-5cb98b5ee847', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.5724334716796875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 14, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6800', 'nonce': 31724}, {'type': 'v1', 'addr': '172.21.15.135:6801', 'nonce': 31724}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6802', 'nonce': 31724}, {'type': 'v1', 'addr': '172.21.15.135:6803', 'nonce': 31724}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6806', 'nonce': 31724}, {'type': 'v1', 'addr': '172.21.15.135:6807', 'nonce': 31724}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6804', 'nonce': 31724}, {'type': 'v1', 'addr': '172.21.15.135:6805', 'nonce': 31724}]}, 'public_addr': '172.21.15.135:6801/31724', 'cluster_addr': '172.21.15.135:6803/31724', 'heartbeat_back_addr': '172.21.15.135:6807/31724', 'heartbeat_front_addr': '172.21.15.135:6805/31724', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'd74d8fef-0211-413b-bff2-f97776fc1faa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 14, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6820', 'nonce': 31734}, {'type': 'v1', 'addr': '172.21.15.135:6822', 'nonce': 31734}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6824', 'nonce': 31734}, {'type': 'v1', 'addr': '172.21.15.135:6826', 'nonce': 31734}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6830', 'nonce': 31734}, {'type': 'v1', 'addr': '172.21.15.135:6831', 'nonce': 31734}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6828', 'nonce': 31734}, {'type': 'v1', 'addr': '172.21.15.135:6829', 'nonce': 31734}]}, 'public_addr': '172.21.15.135:6822/31734', 'cluster_addr': '172.21.15.135:6826/31734', 'heartbeat_back_addr': '172.21.15.135:6831/31734', 'heartbeat_front_addr': '172.21.15.135:6829/31734', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '7dc84c6e-76d8-4f22-8b17-51f62765762d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6808', 'nonce': 31725}, {'type': 'v1', 'addr': '172.21.15.135:6809', 'nonce': 31725}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6810', 'nonce': 31725}, {'type': 'v1', 'addr': '172.21.15.135:6811', 'nonce': 31725}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6814', 'nonce': 31725}, {'type': 'v1', 'addr': '172.21.15.135:6815', 'nonce': 31725}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6812', 'nonce': 31725}, {'type': 'v1', 'addr': '172.21.15.135:6813', 'nonce': 31725}]}, 'public_addr': '172.21.15.135:6809/31725', 'cluster_addr': '172.21.15.135:6811/31725', 'heartbeat_back_addr': '172.21.15.135:6815/31725', 'heartbeat_front_addr': '172.21.15.135:6813/31725', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '7adb3187-aed3-48af-868c-7230e9bfc85e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6816', 'nonce': 31737}, {'type': 'v1', 'addr': '172.21.15.135:6817', 'nonce': 31737}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6818', 'nonce': 31737}, {'type': 'v1', 'addr': '172.21.15.135:6819', 'nonce': 31737}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6825', 'nonce': 31737}, {'type': 'v1', 'addr': '172.21.15.135:6827', 'nonce': 31737}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6821', 'nonce': 31737}, {'type': 'v1', 'addr': '172.21.15.135:6823', 'nonce': 31737}]}, 'public_addr': '172.21.15.135:6817/31737', 'cluster_addr': '172.21.15.135:6819/31737', 'heartbeat_back_addr': '172.21.15.135:6827/31737', 'heartbeat_front_addr': '172.21.15.135:6823/31737', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T20:09:10.683465+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T20:09:10.722423+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T20:09:10.712691+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-04-25T20:09:10.673892+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T20:09:10.768469+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T20:09:10.783897+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T20:09:10.686833+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T20:09:10.739675+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-04-25T20:09:56.628 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-04-25T20:09:56.628 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:56.627+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:56.728 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:56.728+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:56.737 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:56.737 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:56.737 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:09:56.737 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:09:56.737 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:56.747 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T20:09:56.829 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:56.829+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:56.855 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:56.855 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:09:56.855 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:09:56.855 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-04-25T20:09:56.855 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.855 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:30 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'16, mlcod=16'14)", 2024-04-25T20:09:56.855 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:09:56.855 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 39.686830694999998, 2024-04-25T20:09:56.855 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0054087670000000001, 2024-04-25T20:09:56.855 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:56.855 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:56.855 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:56.855 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.856 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:56.856 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:09:56.856 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.856 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.856 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.856 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:56.856 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:09:56.856 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.856 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.856 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.856 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:56.856 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167142+0000", 2024-04-25T20:09:56.856 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999976 2024-04-25T20:09:56.856 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.857 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.857 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:56.857 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167162+0000", 2024-04-25T20:09:56.857 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0109e-05 2024-04-25T20:09:56.857 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.857 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.857 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:56.857 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167188+0000", 2024-04-25T20:09:56.857 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6146e-05 2024-04-25T20:09:56.857 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.857 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.857 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:56.857 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167199+0000", 2024-04-25T20:09:56.857 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0767e-05 2024-04-25T20:09:56.857 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.858 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.858 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:56.858 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167256+0000", 2024-04-25T20:09:56.858 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7392000000000002e-05 2024-04-25T20:09:56.858 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.858 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.858 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:56.858 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167283+0000", 2024-04-25T20:09:56.858 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6577e-05 2024-04-25T20:09:56.858 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.858 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.858 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:56.858 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.172510+0000", 2024-04-25T20:09:56.858 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0052276759999999997 2024-04-25T20:09:56.859 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.859 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.859 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:56.859 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.172553+0000", 2024-04-25T20:09:56.859 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2444999999999998e-05 2024-04-25T20:09:56.859 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.859 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.859 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.859 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.859 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.859 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:31 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-04-25T20:09:56.859 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:09:56.859 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 39.686214616999997, 2024-04-25T20:09:56.859 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0055055479999999999, 2024-04-25T20:09:56.860 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:56.860 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:56.860 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:56.860 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.860 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:56.860 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:09:56.860 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.860 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.860 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.860 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:56.860 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:09:56.860 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.860 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.860 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.860 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:56.861 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:09:56.861 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:56.861 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.861 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.861 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:56.861 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167766+0000", 2024-04-25T20:09:56.861 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.122e-06 2024-04-25T20:09:56.861 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.861 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.861 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:56.861 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167776+0000", 2024-04-25T20:09:56.861 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0515e-05 2024-04-25T20:09:56.861 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.861 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.861 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:56.862 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167782+0000", 2024-04-25T20:09:56.862 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1149999999999999e-06 2024-04-25T20:09:56.862 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.862 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.862 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:56.862 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167837+0000", 2024-04-25T20:09:56.862 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4763999999999999e-05 2024-04-25T20:09:56.862 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.862 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.862 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:56.862 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167855+0000", 2024-04-25T20:09:56.862 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7589e-05 2024-04-25T20:09:56.862 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.862 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.862 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:56.863 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173221+0000", 2024-04-25T20:09:56.863 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.005365903 2024-04-25T20:09:56.863 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.863 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.863 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:56.863 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173266+0000", 2024-04-25T20:09:56.863 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5059999999999999e-05 2024-04-25T20:09:56.863 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.863 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.863 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.863 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.863 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.863 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:32 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-04-25T20:09:56.863 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:09:56.863 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 39.685839432000002, 2024-04-25T20:09:56.863 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0051558239999999998, 2024-04-25T20:09:56.864 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:56.864 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:56.864 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:56.864 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.864 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:56.864 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:09:56.864 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.864 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.864 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.864 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:56.864 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:09:56.864 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.864 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.864 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.864 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:56.865 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:09:56.865 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:56.865 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.865 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.865 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:56.865 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168173+0000", 2024-04-25T20:09:56.865 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7663999999999997e-05 2024-04-25T20:09:56.865 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.865 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.865 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:56.865 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168182+0000", 2024-04-25T20:09:56.865 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0650000000000005e-06 2024-04-25T20:09:56.865 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.865 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.865 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:56.866 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168188+0000", 2024-04-25T20:09:56.866 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0589999999999999e-06 2024-04-25T20:09:56.866 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.866 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.866 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:56.866 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168239+0000", 2024-04-25T20:09:56.866 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1467e-05 2024-04-25T20:09:56.866 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.866 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.866 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:56.866 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168260+0000", 2024-04-25T20:09:56.866 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0531e-05 2024-04-25T20:09:56.866 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.866 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.867 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:56.867 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173268+0000", 2024-04-25T20:09:56.867 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0050087980000000001 2024-04-25T20:09:56.867 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.867 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.867 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:56.867 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173291+0000", 2024-04-25T20:09:56.867 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.268e-05 2024-04-25T20:09:56.867 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.867 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.867 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.867 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.867 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.867 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:33 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'19, mlcod=16'14)", 2024-04-25T20:09:56.867 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:09:56.867 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 39.685329013, 2024-04-25T20:09:56.868 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046697049999999997, 2024-04-25T20:09:56.868 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:56.868 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:56.868 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:56.868 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.868 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:56.868 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:09:56.868 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.868 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.868 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.868 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:56.868 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:09:56.868 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.868 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.868 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.869 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:56.869 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168645+0000", 2024-04-25T20:09:56.869 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:56.869 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.869 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.869 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:56.869 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168651+0000", 2024-04-25T20:09:56.869 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1079999999999998e-06 2024-04-25T20:09:56.869 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.869 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.869 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:56.869 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168660+0000", 2024-04-25T20:09:56.869 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0010000000000007e-06 2024-04-25T20:09:56.869 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.869 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.870 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:56.870 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168666+0000", 2024-04-25T20:09:56.870 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7780000000000004e-06 2024-04-25T20:09:56.870 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.870 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.870 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:56.870 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168701+0000", 2024-04-25T20:09:56.870 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4490999999999999e-05 2024-04-25T20:09:56.870 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.870 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.870 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:56.870 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168729+0000", 2024-04-25T20:09:56.870 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8476999999999999e-05 2024-04-25T20:09:56.870 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.870 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.871 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:56.871 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173294+0000", 2024-04-25T20:09:56.871 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0045650730000000002 2024-04-25T20:09:56.871 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.871 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.871 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:56.871 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173315+0000", 2024-04-25T20:09:56.871 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1245000000000001e-05 2024-04-25T20:09:56.871 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.871 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.872 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.872 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.872 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.872 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:34 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'20, mlcod=16'14)", 2024-04-25T20:09:56.872 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:09:56.872 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 39.684830083999998, 2024-04-25T20:09:56.872 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041943309999999999, 2024-04-25T20:09:56.872 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:56.872 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:56.872 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:56.872 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.872 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:56.872 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:09:56.872 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.872 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.872 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.873 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:56.873 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:09:56.873 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.873 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.873 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.873 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:56.873 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169144+0000", 2024-04-25T20:09:56.873 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:56.873 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.873 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.873 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:56.873 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169150+0000", 2024-04-25T20:09:56.873 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9150000000000001e-06 2024-04-25T20:09:56.873 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.874 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.874 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:56.874 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169159+0000", 2024-04-25T20:09:56.874 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8899999999999996e-06 2024-04-25T20:09:56.874 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.874 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.874 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:56.874 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169169+0000", 2024-04-25T20:09:56.874 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0441e-05 2024-04-25T20:09:56.874 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.874 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.874 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:56.874 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169206+0000", 2024-04-25T20:09:56.874 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6816000000000001e-05 2024-04-25T20:09:56.874 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.875 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.875 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:56.875 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169234+0000", 2024-04-25T20:09:56.875 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8008999999999999e-05 2024-04-25T20:09:56.875 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.875 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.875 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:56.875 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173317+0000", 2024-04-25T20:09:56.875 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0040830160000000001 2024-04-25T20:09:56.875 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.875 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.875 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:56.875 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173339+0000", 2024-04-25T20:09:56.875 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1897000000000001e-05 2024-04-25T20:09:56.875 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.876 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.876 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.876 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.876 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.876 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:35 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'14)", 2024-04-25T20:09:56.876 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:09:56.876 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 39.684337765000002, 2024-04-25T20:09:56.876 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037236529999999999, 2024-04-25T20:09:56.876 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:56.876 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:56.876 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:56.876 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.876 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:56.876 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:09:56.876 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.877 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.877 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.877 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:56.877 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:09:56.877 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.877 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.877 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.877 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:56.877 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169636+0000", 2024-04-25T20:09:56.877 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:56.877 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.877 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.877 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:56.877 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169678+0000", 2024-04-25T20:09:56.877 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1752000000000003e-05 2024-04-25T20:09:56.878 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.878 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.878 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:56.878 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169688+0000", 2024-04-25T20:09:56.878 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0098000000000001e-05 2024-04-25T20:09:56.878 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.878 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.878 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:56.878 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169694+0000", 2024-04-25T20:09:56.878 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2349999999999996e-06 2024-04-25T20:09:56.878 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.878 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.878 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:56.878 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169733+0000", 2024-04-25T20:09:56.878 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8481e-05 2024-04-25T20:09:56.879 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.879 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.879 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:56.879 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169750+0000", 2024-04-25T20:09:56.879 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7509000000000001e-05 2024-04-25T20:09:56.879 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.879 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.879 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:56.879 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173340+0000", 2024-04-25T20:09:56.879 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0035898449999999999 2024-04-25T20:09:56.879 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.879 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.879 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:56.879 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173361+0000", 2024-04-25T20:09:56.879 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.039e-05 2024-04-25T20:09:56.880 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.880 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.880 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.880 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.880 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.880 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:36 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'22, mlcod=16'14)", 2024-04-25T20:09:56.880 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:09:56.880 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 39.683837705000002, 2024-04-25T20:09:56.880 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032472120000000002, 2024-04-25T20:09:56.880 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:56.880 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:56.880 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:56.880 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.880 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:56.880 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:09:56.880 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.881 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.881 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.881 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:56.881 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:09:56.881 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.881 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.881 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.881 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:56.881 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:09:56.881 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:56.881 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.881 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.881 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:56.881 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170152+0000", 2024-04-25T20:09:56.881 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5798e-05 2024-04-25T20:09:56.882 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.882 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.882 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:56.882 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170161+0000", 2024-04-25T20:09:56.882 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.064e-06 2024-04-25T20:09:56.882 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.882 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.882 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:56.882 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170167+0000", 2024-04-25T20:09:56.882 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8390000000000003e-06 2024-04-25T20:09:56.882 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.882 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.882 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:56.882 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170202+0000", 2024-04-25T20:09:56.882 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.5120999999999998e-05 2024-04-25T20:09:56.883 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.883 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.883 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:56.883 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170219+0000", 2024-04-25T20:09:56.883 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6929999999999999e-05 2024-04-25T20:09:56.883 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.883 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.883 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:56.883 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173362+0000", 2024-04-25T20:09:56.883 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0031427619999999999 2024-04-25T20:09:56.883 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.883 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.883 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:56.883 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173384+0000", 2024-04-25T20:09:56.883 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2237000000000001e-05 2024-04-25T20:09:56.883 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.884 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.884 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.884 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.884 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.884 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:40 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'26, mlcod=16'24)", 2024-04-25T20:09:56.884 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:09:56.884 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 39.676799793000001, 2024-04-25T20:09:56.884 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028727330000000001, 2024-04-25T20:09:56.884 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:56.884 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:56.884 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:56.884 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.884 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:56.884 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:09:56.884 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.885 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.885 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.885 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:56.885 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:09:56.885 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.885 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.885 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.885 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:56.885 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177173+0000", 2024-04-25T20:09:56.885 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999976 2024-04-25T20:09:56.885 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.885 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.885 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:56.885 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177186+0000", 2024-04-25T20:09:56.886 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3533000000000001e-05 2024-04-25T20:09:56.886 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.886 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.886 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:56.886 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177209+0000", 2024-04-25T20:09:56.886 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3280000000000001e-05 2024-04-25T20:09:56.886 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.886 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.886 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:56.886 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177220+0000", 2024-04-25T20:09:56.886 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0465000000000001e-05 2024-04-25T20:09:56.886 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.886 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.886 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:56.886 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177264+0000", 2024-04-25T20:09:56.887 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4413999999999999e-05 2024-04-25T20:09:56.887 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.887 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.887 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:56.887 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177289+0000", 2024-04-25T20:09:56.887 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4783999999999999e-05 2024-04-25T20:09:56.887 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.887 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.887 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:56.887 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.179990+0000", 2024-04-25T20:09:56.887 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027012939999999999 2024-04-25T20:09:56.887 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.887 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.887 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:56.887 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.180048+0000", 2024-04-25T20:09:56.888 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7299000000000001e-05 2024-04-25T20:09:56.888 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.888 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.888 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.888 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.888 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.888 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:67 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'41, mlcod=16'39)", 2024-04-25T20:09:56.888 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:09:56.888 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 39.642038300999999, 2024-04-25T20:09:56.888 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064192549999999996, 2024-04-25T20:09:56.888 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:56.888 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:56.888 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:56.888 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.889 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:56.889 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:09:56.889 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.889 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.889 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.889 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:56.889 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:09:56.889 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.889 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.889 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.889 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:56.889 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211935+0000", 2024-04-25T20:09:56.889 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999981 2024-04-25T20:09:56.889 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.889 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.889 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:56.889 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211948+0000", 2024-04-25T20:09:56.890 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3531e-05 2024-04-25T20:09:56.890 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.890 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.890 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:56.890 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211967+0000", 2024-04-25T20:09:56.890 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8326999999999999e-05 2024-04-25T20:09:56.890 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.890 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.890 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:56.890 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211975+0000", 2024-04-25T20:09:56.890 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8589999999999994e-06 2024-04-25T20:09:56.890 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.890 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.890 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:56.891 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212020+0000", 2024-04-25T20:09:56.891 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4626999999999998e-05 2024-04-25T20:09:56.891 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.891 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.891 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:56.891 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212045+0000", 2024-04-25T20:09:56.891 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4508999999999999e-05 2024-04-25T20:09:56.891 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.891 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.891 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:56.891 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218316+0000", 2024-04-25T20:09:56.891 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0062711640000000001 2024-04-25T20:09:56.891 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.892 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.892 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:56.892 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218356+0000", 2024-04-25T20:09:56.892 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9993000000000001e-05 2024-04-25T20:09:56.892 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.892 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.892 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.892 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.892 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.892 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:68 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-04-25T20:09:56.892 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:09:56.892 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 39.641589287000002, 2024-04-25T20:09:56.893 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011231365, 2024-04-25T20:09:56.893 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:56.893 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:56.893 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:56.893 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.893 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:56.893 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:09:56.893 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.893 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.893 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.893 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:56.893 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:09:56.893 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.893 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.893 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.894 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:56.894 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:09:56.894 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:56.894 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.894 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.894 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:56.894 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212392+0000", 2024-04-25T20:09:56.894 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.79e-06 2024-04-25T20:09:56.894 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.894 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.894 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:56.894 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212406+0000", 2024-04-25T20:09:56.894 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3577000000000001e-05 2024-04-25T20:09:56.894 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.895 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.895 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:56.895 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212414+0000", 2024-04-25T20:09:56.895 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6510000000000001e-06 2024-04-25T20:09:56.895 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.895 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.895 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:56.895 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217114+0000", 2024-04-25T20:09:56.895 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047005379999999998 2024-04-25T20:09:56.895 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.895 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.895 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:56.895 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217141+0000", 2024-04-25T20:09:56.895 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6984000000000001e-05 2024-04-25T20:09:56.896 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.896 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.896 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:56.896 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223563+0000", 2024-04-25T20:09:56.896 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064216999999999998 2024-04-25T20:09:56.896 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.896 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.896 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:56.896 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223617+0000", 2024-04-25T20:09:56.896 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4011000000000002e-05 2024-04-25T20:09:56.896 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.896 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.896 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.896 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.896 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.897 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:69 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-04-25T20:09:56.897 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:09:56.897 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 39.641057097000001, 2024-04-25T20:09:56.897 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.010730094000000001, 2024-04-25T20:09:56.897 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:56.897 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:56.897 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:56.897 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.897 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:56.897 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:09:56.897 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.897 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.897 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.897 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:56.897 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:09:56.897 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.898 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.898 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.898 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:56.898 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212917+0000", 2024-04-25T20:09:56.898 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:56.898 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.898 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.898 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:56.898 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212957+0000", 2024-04-25T20:09:56.898 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0692000000000003e-05 2024-04-25T20:09:56.898 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.898 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.898 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:56.898 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212973+0000", 2024-04-25T20:09:56.898 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.575e-05 2024-04-25T20:09:56.899 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.899 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.899 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:56.899 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212980+0000", 2024-04-25T20:09:56.899 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1679999999999998e-06 2024-04-25T20:09:56.899 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.899 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.899 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:56.899 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217449+0000", 2024-04-25T20:09:56.899 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.004468367 2024-04-25T20:09:56.899 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.899 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.899 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:56.899 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217470+0000", 2024-04-25T20:09:56.900 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0749000000000001e-05 2024-04-25T20:09:56.900 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.900 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.900 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:56.900 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223621+0000", 2024-04-25T20:09:56.900 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0061512880000000004 2024-04-25T20:09:56.900 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.900 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.900 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:56.900 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223648+0000", 2024-04-25T20:09:56.900 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6961000000000002e-05 2024-04-25T20:09:56.900 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.900 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.900 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.900 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.900 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.900 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:70 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'44, mlcod=16'39)", 2024-04-25T20:09:56.901 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:09:56.901 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 39.640298369, 2024-04-25T20:09:56.901 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.013553358, 2024-04-25T20:09:56.901 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:56.901 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:56.901 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:56.901 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.901 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:56.901 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:09:56.901 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.901 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.901 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.901 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:56.901 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:09:56.901 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.901 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.901 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.902 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:56.902 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:09:56.902 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:56.902 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.902 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.902 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:56.902 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213682+0000", 2024-04-25T20:09:56.902 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3439999999999997e-06 2024-04-25T20:09:56.902 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.902 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.902 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:56.902 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213693+0000", 2024-04-25T20:09:56.902 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1049999999999999e-05 2024-04-25T20:09:56.902 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.902 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.903 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:56.903 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213699+0000", 2024-04-25T20:09:56.903 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9399999999999999e-06 2024-04-25T20:09:56.903 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.903 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.903 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:56.903 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218086+0000", 2024-04-25T20:09:56.903 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0043871240000000001 2024-04-25T20:09:56.903 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.903 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.903 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:56.903 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218106+0000", 2024-04-25T20:09:56.903 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.997e-05 2024-04-25T20:09:56.903 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.904 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.904 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:56.904 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.227186+0000", 2024-04-25T20:09:56.904 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.009079574 2024-04-25T20:09:56.904 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.904 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.904 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:56.904 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.227230+0000", 2024-04-25T20:09:56.904 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4147999999999998e-05 2024-04-25T20:09:56.904 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.904 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.904 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.904 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.904 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.904 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:106 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-04-25T20:09:56.904 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:09:56.905 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 39.269159268999999, 2024-04-25T20:09:56.905 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002936756, 2024-04-25T20:09:56.905 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:56.905 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:56.905 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:56.905 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.905 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:56.905 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:09:56.905 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.905 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.905 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.905 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:56.905 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:09:56.906 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.906 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.906 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.906 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:56.906 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584815+0000", 2024-04-25T20:09:56.906 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:56.906 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.906 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.906 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:56.906 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584860+0000", 2024-04-25T20:09:56.906 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5161999999999999e-05 2024-04-25T20:09:56.906 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.906 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.906 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:56.907 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584874+0000", 2024-04-25T20:09:56.907 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4464000000000001e-05 2024-04-25T20:09:56.907 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.907 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.907 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:56.907 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584881+0000", 2024-04-25T20:09:56.907 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.409e-06 2024-04-25T20:09:56.907 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.907 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.907 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:56.907 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584915+0000", 2024-04-25T20:09:56.907 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4007000000000002e-05 2024-04-25T20:09:56.907 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.907 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.908 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:56.908 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584933+0000", 2024-04-25T20:09:56.908 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7737999999999999e-05 2024-04-25T20:09:56.908 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.908 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.908 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:56.908 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587717+0000", 2024-04-25T20:09:56.908 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002784711 2024-04-25T20:09:56.908 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.908 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.908 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:56.908 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587752+0000", 2024-04-25T20:09:56.908 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4915e-05 2024-04-25T20:09:56.908 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.908 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.908 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.909 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.909 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.909 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:107 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'69, mlcod=16'64)", 2024-04-25T20:09:56.909 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:09:56.909 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 39.269083967, 2024-04-25T20:09:56.909 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028906270000000002, 2024-04-25T20:09:56.909 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:56.909 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:56.909 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:56.909 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.909 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:56.909 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:09:56.909 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.909 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.909 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.910 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:56.910 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:09:56.910 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.910 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.910 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.910 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:56.910 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584890+0000", 2024-04-25T20:09:56.910 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:56.910 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.910 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.910 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:56.910 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584895+0000", 2024-04-25T20:09:56.910 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2069999999999997e-06 2024-04-25T20:09:56.910 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.911 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.911 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:56.911 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584906+0000", 2024-04-25T20:09:56.911 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.117e-05 2024-04-25T20:09:56.911 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.911 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.911 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:56.911 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584910+0000", 2024-04-25T20:09:56.911 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4190000000000004e-06 2024-04-25T20:09:56.911 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.911 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.911 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:56.911 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585310+0000", 2024-04-25T20:09:56.911 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00040002700000000001 2024-04-25T20:09:56.911 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.912 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.912 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:56.912 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585329+0000", 2024-04-25T20:09:56.912 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8838000000000002e-05 2024-04-25T20:09:56.912 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.912 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.912 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:56.912 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587754+0000", 2024-04-25T20:09:56.912 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024252689999999999 2024-04-25T20:09:56.912 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.912 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.912 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:56.912 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587781+0000", 2024-04-25T20:09:56.912 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7206999999999999e-05 2024-04-25T20:09:56.912 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.912 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.913 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.913 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.913 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.913 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:108 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'70, mlcod=16'64)", 2024-04-25T20:09:56.913 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:09:56.913 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 39.269058815999998, 2024-04-25T20:09:56.913 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028924879999999999, 2024-04-25T20:09:56.913 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:56.913 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:56.913 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:56.913 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.913 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:56.913 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:09:56.913 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.913 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.913 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.914 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:56.914 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:09:56.914 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.914 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.914 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.914 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:56.914 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:09:56.914 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:56.914 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.914 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.914 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:56.914 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584919+0000", 2024-04-25T20:09:56.914 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.557e-06 2024-04-25T20:09:56.914 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.914 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.915 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:56.915 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584925+0000", 2024-04-25T20:09:56.915 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8619999999999999e-06 2024-04-25T20:09:56.915 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.915 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.915 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:56.915 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584931+0000", 2024-04-25T20:09:56.915 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2539999999999998e-06 2024-04-25T20:09:56.915 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.915 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.915 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:56.915 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585550+0000", 2024-04-25T20:09:56.915 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00061821800000000002 2024-04-25T20:09:56.915 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.915 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.915 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:56.916 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585567+0000", 2024-04-25T20:09:56.916 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7499e-05 2024-04-25T20:09:56.916 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.916 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.916 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:56.916 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587783+0000", 2024-04-25T20:09:56.916 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0022156200000000002 2024-04-25T20:09:56.916 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.916 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.916 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:56.916 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587808+0000", 2024-04-25T20:09:56.916 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5729e-05 2024-04-25T20:09:56.916 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.916 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.916 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.916 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.916 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.917 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:112 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'74, mlcod=16'66)", 2024-04-25T20:09:56.917 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:09:56.917 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 39.266901193999999, 2024-04-25T20:09:56.917 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00318724, 2024-04-25T20:09:56.917 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:56.917 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:56.917 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:56.917 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.917 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:56.917 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:09:56.917 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.917 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.917 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.917 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:56.917 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:09:56.918 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.918 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.918 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.918 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:56.918 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587073+0000", 2024-04-25T20:09:56.918 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:56.918 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.918 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.918 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:56.918 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587136+0000", 2024-04-25T20:09:56.918 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2928999999999993e-05 2024-04-25T20:09:56.918 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.918 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.918 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:56.918 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587147+0000", 2024-04-25T20:09:56.919 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1182999999999999e-05 2024-04-25T20:09:56.919 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.919 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.919 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:56.919 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587153+0000", 2024-04-25T20:09:56.919 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9059999999999999e-06 2024-04-25T20:09:56.919 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.919 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.919 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:56.919 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587306+0000", 2024-04-25T20:09:56.919 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000152848 2024-04-25T20:09:56.919 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.919 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.919 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:56.919 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587330+0000", 2024-04-25T20:09:56.920 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4304e-05 2024-04-25T20:09:56.920 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.920 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.920 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:56.920 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.590230+0000", 2024-04-25T20:09:56.920 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029004629999999998 2024-04-25T20:09:56.920 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.920 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.920 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:56.920 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.590261+0000", 2024-04-25T20:09:56.920 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0323000000000001e-05 2024-04-25T20:09:56.920 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.920 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.920 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.921 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.921 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.921 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:118 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'80, mlcod=16'78)", 2024-04-25T20:09:56.921 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:09:56.921 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 39.262114615000002, 2024-04-25T20:09:56.921 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029618959999999999, 2024-04-25T20:09:56.921 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:56.921 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:56.921 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:56.921 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.921 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:56.921 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:09:56.921 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.921 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.921 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.921 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:56.922 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:09:56.922 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.922 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.922 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.922 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:56.922 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591859+0000", 2024-04-25T20:09:56.922 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:09:56.922 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.922 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.922 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:56.922 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591918+0000", 2024-04-25T20:09:56.922 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8842999999999999e-05 2024-04-25T20:09:56.922 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.922 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.922 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:56.923 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591933+0000", 2024-04-25T20:09:56.923 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4600000000000001e-05 2024-04-25T20:09:56.923 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.923 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.923 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:56.923 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591939+0000", 2024-04-25T20:09:56.923 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4559999999999997e-06 2024-04-25T20:09:56.923 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.923 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.923 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:56.923 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591990+0000", 2024-04-25T20:09:56.923 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0318999999999997e-05 2024-04-25T20:09:56.923 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.923 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.923 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:56.923 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.592008+0000", 2024-04-25T20:09:56.924 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8819000000000001e-05 2024-04-25T20:09:56.924 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.924 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.924 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:56.924 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.594784+0000", 2024-04-25T20:09:56.924 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027750850000000001 2024-04-25T20:09:56.924 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.924 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.924 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:56.924 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.594822+0000", 2024-04-25T20:09:56.924 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8522000000000001e-05 2024-04-25T20:09:56.924 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.924 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.924 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.924 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.925 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.925 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:139 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'97, mlcod=17'87)", 2024-04-25T20:09:56.925 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:09:56.925 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 38.728078009999997, 2024-04-25T20:09:56.925 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048546290000000001, 2024-04-25T20:09:56.925 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:56.925 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:56.925 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:56.925 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.925 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:56.925 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:09:56.925 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.925 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.925 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.925 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:56.925 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:09:56.926 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.926 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.926 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.926 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:56.926 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125896+0000", 2024-04-25T20:09:56.926 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:56.926 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.926 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.926 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:56.926 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125908+0000", 2024-04-25T20:09:56.926 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.137e-05 2024-04-25T20:09:56.926 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.926 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.927 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:56.927 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125915+0000", 2024-04-25T20:09:56.927 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3939999999999998e-06 2024-04-25T20:09:56.927 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.927 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.927 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:56.927 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125922+0000", 2024-04-25T20:09:56.927 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1930000000000004e-06 2024-04-25T20:09:56.927 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.927 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.927 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:56.927 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125959+0000", 2024-04-25T20:09:56.927 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6701e-05 2024-04-25T20:09:56.927 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.927 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.928 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:56.928 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125972+0000", 2024-04-25T20:09:56.928 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2578000000000001e-05 2024-04-25T20:09:56.928 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.928 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.928 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:56.928 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130719+0000", 2024-04-25T20:09:56.928 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047475800000000004 2024-04-25T20:09:56.928 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.928 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.928 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:56.928 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130751+0000", 2024-04-25T20:09:56.928 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2283000000000002e-05 2024-04-25T20:09:56.928 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.928 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.928 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.929 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.929 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.929 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:140 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'98, mlcod=17'87)", 2024-04-25T20:09:56.929 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:09:56.929 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 38.727652659, 2024-04-25T20:09:56.929 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048310990000000002, 2024-04-25T20:09:56.929 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:56.929 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:56.929 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:56.929 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.929 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:56.929 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:09:56.929 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.929 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.929 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.929 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:56.930 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:09:56.930 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.930 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.930 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.930 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:56.930 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:09:56.930 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:56.930 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.930 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.930 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:56.930 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126427+0000", 2024-04-25T20:09:56.930 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000105572 2024-04-25T20:09:56.930 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.930 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.931 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:56.931 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126435+0000", 2024-04-25T20:09:56.931 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1920000000000005e-06 2024-04-25T20:09:56.931 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.931 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.931 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:56.931 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126439+0000", 2024-04-25T20:09:56.931 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6950000000000002e-06 2024-04-25T20:09:56.931 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.931 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.931 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:56.931 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126471+0000", 2024-04-25T20:09:56.931 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1699000000000003e-05 2024-04-25T20:09:56.931 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.931 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.932 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:56.932 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126484+0000", 2024-04-25T20:09:56.932 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.343e-05 2024-04-25T20:09:56.932 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.932 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.932 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:56.932 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131131+0000", 2024-04-25T20:09:56.932 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046471480000000003 2024-04-25T20:09:56.932 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.932 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.932 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:56.932 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131153+0000", 2024-04-25T20:09:56.932 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1753e-05 2024-04-25T20:09:56.932 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.932 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.933 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.933 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.933 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.933 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:141 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'99, mlcod=17'87)", 2024-04-25T20:09:56.933 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:09:56.933 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 38.726997715000003, 2024-04-25T20:09:56.933 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.004496847, 2024-04-25T20:09:56.933 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:09:56.933 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:09:56.933 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:09:56.933 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.933 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:09:56.933 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:09:56.933 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.933 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.933 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.934 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:09:56.934 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:09:56.934 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:09:56.934 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.934 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.934 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:09:56.934 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:09:56.934 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:09:56.934 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.934 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.934 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:09:56.934 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126980+0000", 2024-04-25T20:09:56.934 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.619e-06 2024-04-25T20:09:56.934 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.934 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.935 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:09:56.935 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126987+0000", 2024-04-25T20:09:56.935 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7220000000000002e-06 2024-04-25T20:09:56.935 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.935 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.935 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:09:56.935 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126992+0000", 2024-04-25T20:09:56.935 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8679999999999998e-06 2024-04-25T20:09:56.935 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.935 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.935 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:09:56.935 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130772+0000", 2024-04-25T20:09:56.935 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037803120000000001 2024-04-25T20:09:56.935 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.935 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.936 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:09:56.936 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130786+0000", 2024-04-25T20:09:56.936 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3592e-05 2024-04-25T20:09:56.936 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.936 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.936 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:09:56.936 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131451+0000", 2024-04-25T20:09:56.936 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000665206 2024-04-25T20:09:56.936 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:09:56.936 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:09:56.936 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:09:56.936 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131474+0000", 2024-04-25T20:09:56.936 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2957999999999999e-05 2024-04-25T20:09:56.936 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.936 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.937 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.937 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:09:56.937 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:09:56.937 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:56.938 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T20:09:56.939 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:56.938+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:57.039 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:57.039+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:57.086 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:57.086 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:57.086 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:09:57.086 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:57.095 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T20:09:57.140 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:57.140+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:57.240 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:57.240+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:57.241 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:57.241 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:57.241 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:09:57.242 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:09:57.242 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:57.250 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T20:09:57.294 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is unset 2024-04-25T20:09:57.342 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:57.341+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:57.361 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:57.361 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:09:57.361 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:09:57.361 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:09:57.361 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:57.370 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T20:09:57.442 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:57.441+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:57.476 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:57.476 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:57.476 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:09:57.476 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:57.485 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T20:09:57.543 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:57.542+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:57.594 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:57.595 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:09:57.595 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:09:57.595 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:09:57.595 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:57.606 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T20:09:57.650 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:57.650+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:57.714 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:09:57.715 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:09:57.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:09:57.715 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:09:57.715 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:09:57.724 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T20:09:57.769 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:57.768+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:57.838 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:57.838 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:57.838 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:09:57.838 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:57.845 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:57.845+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:57.847 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T20:09:57.946 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:57.946+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:57.989 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:57.989 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:57.989 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:09:57.990 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:09:57.990 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:57.998 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T20:09:58.047 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:58.047+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:58.139 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:58.140 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:09:58.140 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:09:58.140 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:09:58.140 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:58.148 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T20:09:58.148 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:58.147+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:58.229 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T20:09:58.249 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:58.248+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:58.289 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:58.289 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:58.289 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:09:58.289 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:58.298 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T20:09:58.349 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:58.349+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:58.354 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:09:58.354 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:09:58.371 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:09:58.371 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:09:58.391 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:09:58.391 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:09:58.408 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:09:58.408 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:09:58.425 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:09:58.425 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:09:58.439 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:09:58.439 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:09:58.439 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:09:58.439 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:09:58.439 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:09:58.443 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:09:58.443 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:09:58.448 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T20:09:58.451 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:58.450+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:58.459 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:09:58.460 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:09:58.476 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:09:58.476 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:09:58.549 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:58.550 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:58.550 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:58.550+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:58.558 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T20:09:58.651 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:09:58.651+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:09:58.659 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:58.659 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:58.668 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T20:09:58.752 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:58.752+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:58.769 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:58.769 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:58.778 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T20:09:58.854 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:58.853+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:58.878 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:58.879 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:58.887 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T20:09:58.955 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:58.955+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:59.029 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:59.029 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:59.038 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T20:09:59.082 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:59.081+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:59.138 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:59.139 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:59.148 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T20:09:59.190 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:59.189+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:59.259 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:09:59.258+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:09:59.290 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:59.290 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:59.299 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T20:09:59.308 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-25T20:09:59.359 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:09:59.359+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:09:59.411 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:59.412 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:59.421 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T20:09:59.461 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:59.461+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:59.536 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:59.536 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:59.545 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T20:09:59.589 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:09:59.589+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:09:59.662 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:09:59.662+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:09:59.665 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:59.665 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:59.675 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T20:09:59.763 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:09:59.763+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:09:59.822 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:59.823 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:59.832 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T20:09:59.864 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:09:59.864+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:09:59.965 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:09:59.965+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:09:59.978 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:09:59.978 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:09:59.987 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T20:10:00.066 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:00.066+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:00.136 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:00.136 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:00.145 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T20:10:00.167 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:00.167+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:00.252 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:00.252 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:00.261 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T20:10:00.267 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:00.267+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:00.338 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is unset 2024-04-25T20:10:00.368 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:00.368+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:00.376 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:00.376 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:00.386 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T20:10:00.470 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:00.470+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:00.488 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T20:10:00.541 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:00.541 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:00.551 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T20:10:00.571 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:00.570+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:00.622 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:10:00.622 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:10:00.639 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:10:00.640 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:10:00.658 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:10:00.658 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:10:00.668 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:00.668 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:00.672 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:00.672+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:00.676 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:10:00.676 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:10:00.679 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T20:10:00.697 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:10:00.697 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:10:00.718 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:10:00.718 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:10:00.735 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:10:00.736 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:10:00.754 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:10:00.754 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:10:00.773 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:00.773+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:00.813 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:00.813 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:00.813 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:00.813 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:00.813 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:00.823 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T20:10:00.875 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:00.875+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:00.933 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:00.933 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:00.934 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:00.934 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:10:00.934 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:00.943 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T20:10:00.987 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:00.986+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:01.058 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:01.058 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:01.058 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:10:01.058 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:01.067 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T20:10:01.108 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:01.107+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:01.177 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:01.177+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:01.210 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:01.210 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:01.210 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:01.210 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:01.210 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:01.218 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T20:10:01.278 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:01.277+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:01.320 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:01.320 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:01.320 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:01.320 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:10:01.320 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:01.329 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T20:10:01.413 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:01.412+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:01.471 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:01.471 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:01.472 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:10:01.472 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:01.480 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:01.479+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:01.481 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T20:10:01.581 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:01.580+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:01.622 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:01.622 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:01.622 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:01.622 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:01.623 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:01.629 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6, 7] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-25T20:10:01.629 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T20:10:01.629 INFO:tasks.thrashosds.thrasher:Reweighting osd 2 to 0.7264274065563712 2024-04-25T20:10:01.629 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 0.7264274065563712 2024-04-25T20:10:01.631 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T20:10:01.681 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:01.681+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:01.732 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:01.732 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:01.732 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:01.732 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:10:01.732 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:01.741 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T20:10:01.783 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:01.782+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:01.883 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:01.883+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:01.884 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:01.884 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:01.884 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:10:01.884 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:01.893 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T20:10:01.985 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:01.984+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:01.995 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:01.995 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:01.995 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:01.995 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:01.995 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:02.004 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T20:10:02.085 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:02.085+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:02.146 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:02.146 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:02.146 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:02.146 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:10:02.146 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:02.154 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T20:10:02.187 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:02.186+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:02.256 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:02.256 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:02.256 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:10:02.256 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:02.264 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T20:10:02.309 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:02.308+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:02.335 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.2 to 0.726427 (b9f7) 2024-04-25T20:10:02.353 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-25T20:10:02.368 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:02.368 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:02.368 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:02.368 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:02.368 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:02.377 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T20:10:02.390 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:02.390+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:02.490 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:02.489+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:02.521 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:02.521 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:02.521 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:02.521 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [ 2024-04-25T20:10:02.521 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.521 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:02.521 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:10:02.522 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 45.737203303000001, 2024-04-25T20:10:02.522 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.34154860100000001, 2024-04-25T20:10:02.522 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:02.522 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:02.522 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:02.522 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:02.522 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:02.522 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 1 2024-04-25T20:10:02.522 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.522 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:02.522 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.522 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:02.522 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:10:02.522 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.523 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.523 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.523 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:02.523 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:10:02.523 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.523 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.523 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.523 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:02.523 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782261+0000", 2024-04-25T20:10:02.523 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999986 2024-04-25T20:10:02.523 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.523 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.523 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:02.523 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782268+0000", 2024-04-25T20:10:02.524 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.8140000000000001e-06 2024-04-25T20:10:02.524 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.524 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.524 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:02.524 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782279+0000", 2024-04-25T20:10:02.524 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0665999999999999e-05 2024-04-25T20:10:02.524 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.524 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.524 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:02.524 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782290+0000", 2024-04-25T20:10:02.524 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.1343e-05 2024-04-25T20:10:02.524 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.524 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.524 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:02.524 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782340+0000", 2024-04-25T20:10:02.525 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.9311999999999998e-05 2024-04-25T20:10:02.525 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.525 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.525 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for peered", 2024-04-25T20:10:02.525 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782348+0000", 2024-04-25T20:10:02.525 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.0690000000000002e-06 2024-04-25T20:10:02.525 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.525 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.525 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:02.525 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120520+0000", 2024-04-25T20:10:02.525 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.33817188100000001 2024-04-25T20:10:02.525 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.525 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.525 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:02.525 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120656+0000", 2024-04-25T20:10:02.526 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000135897 2024-04-25T20:10:02.526 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.526 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.526 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:02.526 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120802+0000", 2024-04-25T20:10:02.526 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00014691199999999999 2024-04-25T20:10:02.526 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.526 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.526 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:02.526 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.122368+0000", 2024-04-25T20:10:02.526 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0015651040000000001 2024-04-25T20:10:02.526 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.526 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.526 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:02.527 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123729+0000", 2024-04-25T20:10:02.527 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0013610499999999999 2024-04-25T20:10:02.527 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.527 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.527 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:02.527 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123769+0000", 2024-04-25T20:10:02.527 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0719000000000001e-05 2024-04-25T20:10:02.527 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.527 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.527 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:02.527 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123811+0000", 2024-04-25T20:10:02.527 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1267999999999999e-05 2024-04-25T20:10:02.527 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.527 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:02.528 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.528 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.528 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.528 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:67 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:02.528 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:10:02.528 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 45.307784415999997, 2024-04-25T20:10:02.528 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.006947651, 2024-04-25T20:10:02.528 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:02.528 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:02.528 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:02.528 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:02.528 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:02.528 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 67 2024-04-25T20:10:02.528 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.528 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:02.529 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.529 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:02.529 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:10:02.529 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.529 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.529 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.529 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:02.529 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:10:02.529 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.529 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.529 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.529 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:02.529 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211680+0000", 2024-04-25T20:10:02.529 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:02.529 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.530 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.530 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:02.530 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211684+0000", 2024-04-25T20:10:02.530 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4249999999999998e-06 2024-04-25T20:10:02.530 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.530 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.530 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:02.530 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211690+0000", 2024-04-25T20:10:02.530 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.7360000000000004e-06 2024-04-25T20:10:02.530 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.530 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.530 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:02.530 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211694+0000", 2024-04-25T20:10:02.530 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4860000000000002e-06 2024-04-25T20:10:02.531 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.531 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.531 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:02.531 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211727+0000", 2024-04-25T20:10:02.531 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3445999999999998e-05 2024-04-25T20:10:02.531 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.531 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.531 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:02.531 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211778+0000", 2024-04-25T20:10:02.531 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1050000000000001e-05 2024-04-25T20:10:02.531 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.531 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.531 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:02.531 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211851+0000", 2024-04-25T20:10:02.531 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2914e-05 2024-04-25T20:10:02.531 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.532 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.532 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:02.532 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213824+0000", 2024-04-25T20:10:02.532 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0019726520000000001 2024-04-25T20:10:02.532 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.532 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.532 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:02.532 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218579+0000", 2024-04-25T20:10:02.532 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0047553159999999999 2024-04-25T20:10:02.532 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.532 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.532 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:02.532 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218607+0000", 2024-04-25T20:10:02.532 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8144999999999999e-05 2024-04-25T20:10:02.532 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.533 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.533 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:02.533 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218629+0000", 2024-04-25T20:10:02.533 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.1044e-05 2024-04-25T20:10:02.533 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.533 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:02.533 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.533 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.533 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.533 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:68 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:02.533 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:02.533 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 45.307766075000004, 2024-04-25T20:10:02.533 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012255596000000001, 2024-04-25T20:10:02.533 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:02.534 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:02.534 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:02.534 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:02.534 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:02.534 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 68 2024-04-25T20:10:02.534 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.534 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:02.534 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.534 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:02.534 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:02.534 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.534 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.534 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.534 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:02.535 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:02.535 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.535 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.535 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.535 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:02.535 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:02.535 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:02.535 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.535 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.535 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:02.535 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211701+0000", 2024-04-25T20:10:02.535 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2029999999999999e-06 2024-04-25T20:10:02.535 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.535 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.536 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:02.536 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211705+0000", 2024-04-25T20:10:02.536 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.2320000000000003e-06 2024-04-25T20:10:02.536 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.536 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.536 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:02.536 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211708+0000", 2024-04-25T20:10:02.536 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.6290000000000002e-06 2024-04-25T20:10:02.536 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.536 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.536 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:02.536 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212136+0000", 2024-04-25T20:10:02.536 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00042764099999999998 2024-04-25T20:10:02.536 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.536 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.537 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:02.537 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212212+0000", 2024-04-25T20:10:02.537 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.6315000000000004e-05 2024-04-25T20:10:02.537 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.537 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.537 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:02.537 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212306+0000", 2024-04-25T20:10:02.537 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.3556999999999999e-05 2024-04-25T20:10:02.537 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.537 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.537 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:02.537 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213829+0000", 2024-04-25T20:10:02.537 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001523256 2024-04-25T20:10:02.537 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.537 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.538 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:02.538 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223904+0000", 2024-04-25T20:10:02.538 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010075387 2024-04-25T20:10:02.538 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.538 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.538 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:02.538 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223934+0000", 2024-04-25T20:10:02.538 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9988000000000002e-05 2024-04-25T20:10:02.538 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.538 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.538 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:02.538 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223955+0000", 2024-04-25T20:10:02.538 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0630999999999999e-05 2024-04-25T20:10:02.538 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.539 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:02.539 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.539 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.539 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.539 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:69 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:02.539 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:02.539 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 45.307744339999999, 2024-04-25T20:10:02.539 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012316863000000001, 2024-04-25T20:10:02.539 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:02.539 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:02.539 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:02.539 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:02.539 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:02.539 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 69 2024-04-25T20:10:02.539 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.540 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:02.540 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.540 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:02.540 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:02.540 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.540 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.540 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.540 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:02.540 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:02.540 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.540 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.540 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.540 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:02.540 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:02.540 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:02.541 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.541 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.541 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:02.541 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211762+0000", 2024-04-25T20:10:02.541 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1703999999999999e-05 2024-04-25T20:10:02.541 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.541 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.541 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:02.541 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211768+0000", 2024-04-25T20:10:02.541 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.3430000000000003e-06 2024-04-25T20:10:02.541 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.541 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.541 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:02.542 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211771+0000", 2024-04-25T20:10:02.542 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.18e-06 2024-04-25T20:10:02.542 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.542 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.542 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:02.542 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212652+0000", 2024-04-25T20:10:02.542 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00088095800000000004 2024-04-25T20:10:02.542 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.542 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.542 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:02.542 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212703+0000", 2024-04-25T20:10:02.542 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.0655999999999997e-05 2024-04-25T20:10:02.542 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.542 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.542 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:02.543 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212781+0000", 2024-04-25T20:10:02.543 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.8852999999999999e-05 2024-04-25T20:10:02.543 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.543 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.543 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:02.543 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.214131+0000", 2024-04-25T20:10:02.543 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0013496039999999999 2024-04-25T20:10:02.543 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.543 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.543 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:02.543 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223986+0000", 2024-04-25T20:10:02.543 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.009854804 2024-04-25T20:10:02.543 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.543 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.543 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:02.544 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.224017+0000", 2024-04-25T20:10:02.544 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.1068999999999997e-05 2024-04-25T20:10:02.544 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.544 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.544 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:02.544 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.224038+0000", 2024-04-25T20:10:02.544 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0965e-05 2024-04-25T20:10:02.544 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.544 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:02.544 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.544 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.544 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.544 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:70 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:02.544 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:02.544 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 45.307689259, 2024-04-25T20:10:02.544 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.015776348999999999, 2024-04-25T20:10:02.545 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:02.545 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:02.545 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:02.545 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:02.545 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:02.545 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 70 2024-04-25T20:10:02.545 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.545 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:02.545 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.545 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:02.545 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:02.545 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.545 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.545 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.545 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:02.546 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:02.546 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.546 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.546 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.546 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:02.546 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:02.546 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:02.546 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.546 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.546 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:02.546 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211778+0000", 2024-04-25T20:10:02.546 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5330000000000001e-06 2024-04-25T20:10:02.546 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.546 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.546 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:02.547 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211782+0000", 2024-04-25T20:10:02.547 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4079999999999998e-06 2024-04-25T20:10:02.547 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.547 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.547 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:02.547 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211786+0000", 2024-04-25T20:10:02.547 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.6190000000000002e-06 2024-04-25T20:10:02.547 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.547 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.547 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:02.547 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213374+0000", 2024-04-25T20:10:02.547 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0015881859999999999 2024-04-25T20:10:02.547 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.547 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.548 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:02.548 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213478+0000", 2024-04-25T20:10:02.548 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000103464 2024-04-25T20:10:02.548 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.548 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.548 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:02.548 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213601+0000", 2024-04-25T20:10:02.548 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00012324300000000001 2024-04-25T20:10:02.548 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.548 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.548 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:02.548 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.214788+0000", 2024-04-25T20:10:02.548 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011868569999999999 2024-04-25T20:10:02.548 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.548 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.548 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:02.549 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227504+0000", 2024-04-25T20:10:02.549 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012715749 2024-04-25T20:10:02.549 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.549 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.549 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:02.549 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227532+0000", 2024-04-25T20:10:02.549 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.7937e-05 2024-04-25T20:10:02.549 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.549 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.549 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:02.549 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227552+0000", 2024-04-25T20:10:02.549 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0772999999999999e-05 2024-04-25T20:10:02.549 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.549 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:02.550 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.550 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.550 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.550 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:131 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:02.550 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:10:02.550 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 44.398679414, 2024-04-25T20:10:02.550 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0086158810000000006, 2024-04-25T20:10:02.550 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:02.550 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:02.550 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:02.550 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:02.550 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:02.550 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 131 2024-04-25T20:10:02.550 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.550 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:02.551 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.551 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:02.551 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:10:02.551 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.551 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.551 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.551 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:02.551 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:10:02.551 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.551 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.551 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.551 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:02.551 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120785+0000", 2024-04-25T20:10:02.551 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:02.551 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.552 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.552 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:02.552 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120880+0000", 2024-04-25T20:10:02.552 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.4925000000000003e-05 2024-04-25T20:10:02.552 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.552 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.552 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:02.552 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120891+0000", 2024-04-25T20:10:02.552 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0915e-05 2024-04-25T20:10:02.552 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.552 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.552 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:02.552 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120895+0000", 2024-04-25T20:10:02.552 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.8070000000000001e-06 2024-04-25T20:10:02.552 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.553 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.553 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:02.553 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121640+0000", 2024-04-25T20:10:02.553 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00074468500000000003 2024-04-25T20:10:02.553 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.553 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.553 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:02.553 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121684+0000", 2024-04-25T20:10:02.553 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4452000000000001e-05 2024-04-25T20:10:02.553 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.553 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.553 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:02.553 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121753+0000", 2024-04-25T20:10:02.553 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8508000000000002e-05 2024-04-25T20:10:02.553 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.554 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.554 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:02.554 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123618+0000", 2024-04-25T20:10:02.554 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0018655099999999999 2024-04-25T20:10:02.554 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.554 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.554 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:02.554 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129248+0000", 2024-04-25T20:10:02.554 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0056295540000000002 2024-04-25T20:10:02.554 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.554 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.554 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:02.554 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129368+0000", 2024-04-25T20:10:02.554 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000120244 2024-04-25T20:10:02.555 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.555 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.555 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:02.555 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129402+0000", 2024-04-25T20:10:02.555 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3812000000000001e-05 2024-04-25T20:10:02.555 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.555 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:02.555 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.555 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.555 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.555 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:132 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:02.555 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:02.555 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 44.398565204000001, 2024-04-25T20:10:02.555 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0085898169999999996, 2024-04-25T20:10:02.555 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:02.556 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:02.556 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:02.556 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:02.556 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:02.556 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 132 2024-04-25T20:10:02.556 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.556 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:02.556 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.556 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:02.556 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:02.556 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.556 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.556 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.556 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:02.556 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:02.557 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.557 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.557 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.557 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:02.557 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:02.557 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:02.557 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.557 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.557 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:02.557 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120902+0000", 2024-04-25T20:10:02.557 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.6129999999999998e-06 2024-04-25T20:10:02.557 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.557 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.557 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:02.557 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120906+0000", 2024-04-25T20:10:02.558 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6569999999999999e-06 2024-04-25T20:10:02.558 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.558 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.558 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:02.558 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120909+0000", 2024-04-25T20:10:02.558 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5550000000000001e-06 2024-04-25T20:10:02.558 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.558 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.558 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:02.558 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122098+0000", 2024-04-25T20:10:02.558 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011898049999999999 2024-04-25T20:10:02.558 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.558 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.558 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:02.559 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122172+0000", 2024-04-25T20:10:02.559 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3917000000000004e-05 2024-04-25T20:10:02.559 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.559 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.559 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:02.559 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122288+0000", 2024-04-25T20:10:02.559 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00011566 2024-04-25T20:10:02.559 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.559 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.559 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:02.559 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124519+0000", 2024-04-25T20:10:02.559 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0022308359999999999 2024-04-25T20:10:02.559 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.559 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.559 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:02.560 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129414+0000", 2024-04-25T20:10:02.560 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048953720000000003 2024-04-25T20:10:02.560 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.560 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.560 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:02.560 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129458+0000", 2024-04-25T20:10:02.560 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.3426000000000001e-05 2024-04-25T20:10:02.560 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.560 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.560 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:02.560 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129490+0000", 2024-04-25T20:10:02.560 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2329999999999997e-05 2024-04-25T20:10:02.560 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.560 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:02.560 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.561 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.561 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.561 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:133 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:02.561 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:02.561 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 44.398511335000002, 2024-04-25T20:10:02.561 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0086066270000000004, 2024-04-25T20:10:02.561 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:02.561 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:02.561 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:02.561 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:02.561 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:02.561 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 133 2024-04-25T20:10:02.561 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.561 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:02.562 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.562 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:02.562 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:02.562 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.562 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.562 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.562 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:02.562 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:02.562 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.562 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.562 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.562 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:02.562 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:02.562 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:02.562 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.563 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.563 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:02.563 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120956+0000", 2024-04-25T20:10:02.563 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5289999999999998e-06 2024-04-25T20:10:02.563 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.563 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.563 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:02.563 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120962+0000", 2024-04-25T20:10:02.563 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.9490000000000001e-06 2024-04-25T20:10:02.563 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.563 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.563 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:02.563 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120965+0000", 2024-04-25T20:10:02.563 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.1860000000000001e-06 2024-04-25T20:10:02.563 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.564 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.564 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:02.564 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122666+0000", 2024-04-25T20:10:02.564 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0017004500000000001 2024-04-25T20:10:02.564 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.564 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.564 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:02.564 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122741+0000", 2024-04-25T20:10:02.564 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.5297999999999996e-05 2024-04-25T20:10:02.564 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.564 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.564 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:02.564 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122858+0000", 2024-04-25T20:10:02.564 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00011723899999999999 2024-04-25T20:10:02.564 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.565 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.565 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:02.565 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124625+0000", 2024-04-25T20:10:02.565 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0017670100000000001 2024-04-25T20:10:02.565 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.565 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.565 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:02.565 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129497+0000", 2024-04-25T20:10:02.565 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048713580000000001 2024-04-25T20:10:02.565 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.565 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.565 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:02.565 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129532+0000", 2024-04-25T20:10:02.565 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5704999999999998e-05 2024-04-25T20:10:02.565 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.566 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.566 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:02.566 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129561+0000", 2024-04-25T20:10:02.566 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8283e-05 2024-04-25T20:10:02.566 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.566 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:02.566 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.566 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.566 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.566 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:134 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:02.566 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:02.566 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 44.398409137000002, 2024-04-25T20:10:02.566 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0085771179999999999, 2024-04-25T20:10:02.566 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:02.566 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:02.567 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:02.567 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:02.567 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:02.567 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 134 2024-04-25T20:10:02.567 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.567 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:02.567 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.567 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:02.567 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:02.567 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.567 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.567 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.567 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:02.567 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:02.567 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.568 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.568 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.568 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:02.568 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:02.568 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:02.568 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.568 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.568 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:02.568 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121157+0000", 2024-04-25T20:10:02.568 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00010104599999999999 2024-04-25T20:10:02.568 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.568 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.568 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:02.568 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121189+0000", 2024-04-25T20:10:02.568 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2589000000000003e-05 2024-04-25T20:10:02.569 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.569 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.569 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:02.569 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121195+0000", 2024-04-25T20:10:02.569 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.083e-06 2024-04-25T20:10:02.569 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.569 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.569 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:02.569 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123136+0000", 2024-04-25T20:10:02.569 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0019409780000000001 2024-04-25T20:10:02.569 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.569 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.569 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:02.569 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123181+0000", 2024-04-25T20:10:02.569 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4267999999999997e-05 2024-04-25T20:10:02.570 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.570 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.570 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:02.570 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123247+0000", 2024-04-25T20:10:02.570 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.6086000000000004e-05 2024-04-25T20:10:02.570 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.570 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.570 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:02.570 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125639+0000", 2024-04-25T20:10:02.570 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.002392 2024-04-25T20:10:02.570 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.570 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.570 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:02.570 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129567+0000", 2024-04-25T20:10:02.571 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0039282170000000003 2024-04-25T20:10:02.571 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.571 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.571 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:02.571 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129605+0000", 2024-04-25T20:10:02.571 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.7660999999999997e-05 2024-04-25T20:10:02.571 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.571 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.571 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:02.571 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129633+0000", 2024-04-25T20:10:02.571 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8649999999999998e-05 2024-04-25T20:10:02.571 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.571 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:02.571 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.571 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.572 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.572 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:135 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:02.572 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:10:02.572 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 44.398250130999998, 2024-04-25T20:10:02.572 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010898453000000001, 2024-04-25T20:10:02.572 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:02.572 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:02.572 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:02.572 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:02.572 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:02.572 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 135 2024-04-25T20:10:02.572 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.572 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:02.572 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.573 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:02.573 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:10:02.573 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.573 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.573 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.573 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:02.573 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:10:02.573 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.573 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.573 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.573 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:02.573 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121214+0000", 2024-04-25T20:10:02.573 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:02.573 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.573 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.574 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:02.574 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121218+0000", 2024-04-25T20:10:02.574 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6619999999999998e-06 2024-04-25T20:10:02.574 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.574 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.574 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:02.574 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121223+0000", 2024-04-25T20:10:02.574 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1070000000000004e-06 2024-04-25T20:10:02.574 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.574 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.574 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:02.574 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121227+0000", 2024-04-25T20:10:02.574 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2820000000000001e-06 2024-04-25T20:10:02.574 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.574 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.575 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:02.575 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123649+0000", 2024-04-25T20:10:02.575 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0024221970000000001 2024-04-25T20:10:02.575 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.575 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.575 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:02.575 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123723+0000", 2024-04-25T20:10:02.575 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.4386000000000002e-05 2024-04-25T20:10:02.575 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.575 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.575 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:02.575 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123844+0000", 2024-04-25T20:10:02.575 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000121249 2024-04-25T20:10:02.575 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.575 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.576 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:02.576 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125670+0000", 2024-04-25T20:10:02.576 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001825901 2024-04-25T20:10:02.576 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.576 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.576 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:02.576 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132043+0000", 2024-04-25T20:10:02.576 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0063724539999999996 2024-04-25T20:10:02.576 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.576 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.576 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:02.576 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132084+0000", 2024-04-25T20:10:02.576 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0963000000000001e-05 2024-04-25T20:10:02.576 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.576 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.577 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:02.577 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132114+0000", 2024-04-25T20:10:02.577 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9927e-05 2024-04-25T20:10:02.577 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.577 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:02.577 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.577 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.577 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.577 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:136 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:02.577 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:10:02.577 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 44.398234604999999, 2024-04-25T20:10:02.577 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010952513000000001, 2024-04-25T20:10:02.577 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:02.577 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:02.577 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:02.578 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:02.578 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:02.578 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 136 2024-04-25T20:10:02.578 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.578 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:02.578 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.578 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:02.578 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:10:02.578 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.578 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.578 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.578 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:02.578 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:10:02.578 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.579 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.579 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.579 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:02.579 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121230+0000", 2024-04-25T20:10:02.579 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:02.579 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.579 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.579 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:02.579 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121236+0000", 2024-04-25T20:10:02.579 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.7320000000000003e-06 2024-04-25T20:10:02.579 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.579 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.579 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:02.579 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121242+0000", 2024-04-25T20:10:02.579 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.7810000000000002e-06 2024-04-25T20:10:02.580 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.580 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.580 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:02.580 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121246+0000", 2024-04-25T20:10:02.580 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.639e-06 2024-04-25T20:10:02.580 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.580 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.580 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:02.580 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124150+0000", 2024-04-25T20:10:02.580 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.002903816 2024-04-25T20:10:02.580 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.580 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.580 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:02.580 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124195+0000", 2024-04-25T20:10:02.580 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.5042e-05 2024-04-25T20:10:02.581 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.581 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.581 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:02.581 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124266+0000", 2024-04-25T20:10:02.581 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0234999999999994e-05 2024-04-25T20:10:02.581 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.581 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.581 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:02.581 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125711+0000", 2024-04-25T20:10:02.581 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0014458800000000001 2024-04-25T20:10:02.581 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.581 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.581 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:02.581 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132120+0000", 2024-04-25T20:10:02.581 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0064084399999999996 2024-04-25T20:10:02.582 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.582 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.582 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:02.582 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132155+0000", 2024-04-25T20:10:02.582 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4938000000000003e-05 2024-04-25T20:10:02.582 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.582 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.582 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:02.582 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132183+0000", 2024-04-25T20:10:02.582 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8360999999999998e-05 2024-04-25T20:10:02.582 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.582 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:02.582 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.582 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.583 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.583 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:02.583 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:02.583 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 44.398184872999998, 2024-04-25T20:10:02.583 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.01097684, 2024-04-25T20:10:02.583 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:02.583 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:02.583 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:02.583 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:02.583 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:02.583 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 137 2024-04-25T20:10:02.583 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.583 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:02.583 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.583 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:02.584 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:02.584 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.584 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.584 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.584 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:02.584 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:02.584 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.584 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.584 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.584 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:02.584 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:02.584 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:02.584 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.584 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.584 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:02.585 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121328+0000", 2024-04-25T20:10:02.585 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.7908999999999998e-05 2024-04-25T20:10:02.585 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.585 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.585 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:02.585 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121337+0000", 2024-04-25T20:10:02.585 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.9369999999999993e-06 2024-04-25T20:10:02.585 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.585 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.585 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:02.585 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121343+0000", 2024-04-25T20:10:02.585 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.8089999999999998e-06 2024-04-25T20:10:02.585 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.585 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.585 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:02.586 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124656+0000", 2024-04-25T20:10:02.586 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0033133839999999999 2024-04-25T20:10:02.586 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.586 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.586 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:02.586 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124730+0000", 2024-04-25T20:10:02.586 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3565000000000005e-05 2024-04-25T20:10:02.586 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.586 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.586 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:02.586 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124846+0000", 2024-04-25T20:10:02.586 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000116416 2024-04-25T20:10:02.586 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.586 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.587 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:02.587 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126756+0000", 2024-04-25T20:10:02.587 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001909991 2024-04-25T20:10:02.587 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.587 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.587 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:02.587 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132190+0000", 2024-04-25T20:10:02.587 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0054343880000000001 2024-04-25T20:10:02.587 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.587 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.587 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:02.587 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132227+0000", 2024-04-25T20:10:02.587 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6875000000000002e-05 2024-04-25T20:10:02.587 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.588 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.588 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:02.588 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132257+0000", 2024-04-25T20:10:02.588 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.0046e-05 2024-04-25T20:10:02.588 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.588 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:02.588 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.588 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.588 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.588 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:02.588 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:02.588 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 44.398114915000001, 2024-04-25T20:10:02.588 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010978653999999999, 2024-04-25T20:10:02.588 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:02.588 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:02.589 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:02.589 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:02.589 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:02.589 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 138 2024-04-25T20:10:02.589 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.589 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:02.589 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.589 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:02.589 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:02.589 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.589 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.589 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.589 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:02.589 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:02.589 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.590 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.590 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.590 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:02.590 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:02.590 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:02.590 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.590 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.590 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:02.590 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121355+0000", 2024-04-25T20:10:02.590 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.9869999999999999e-06 2024-04-25T20:10:02.590 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.590 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.590 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:02.590 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121361+0000", 2024-04-25T20:10:02.590 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.4620000000000001e-06 2024-04-25T20:10:02.591 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.591 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.591 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:02.591 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121373+0000", 2024-04-25T20:10:02.591 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.2035000000000001e-05 2024-04-25T20:10:02.591 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.591 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.591 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:02.591 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125267+0000", 2024-04-25T20:10:02.591 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0038939230000000001 2024-04-25T20:10:02.591 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.591 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.591 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:02.591 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125309+0000", 2024-04-25T20:10:02.591 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1690999999999998e-05 2024-04-25T20:10:02.592 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.592 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.592 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:02.592 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125385+0000", 2024-04-25T20:10:02.592 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.6162999999999996e-05 2024-04-25T20:10:02.592 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.592 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.592 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:02.592 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126798+0000", 2024-04-25T20:10:02.592 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0014124680000000001 2024-04-25T20:10:02.592 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.592 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.592 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:02.592 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132264+0000", 2024-04-25T20:10:02.592 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0054661689999999999 2024-04-25T20:10:02.593 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.593 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.593 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:02.593 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132299+0000", 2024-04-25T20:10:02.593 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5272999999999999e-05 2024-04-25T20:10:02.593 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.593 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.593 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:02.593 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132329+0000", 2024-04-25T20:10:02.593 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.0080999999999999e-05 2024-04-25T20:10:02.593 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.593 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:02.593 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.593 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.593 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.594 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:02.594 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:10:02.594 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 44.398084339999997, 2024-04-25T20:10:02.594 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011015853000000001, 2024-04-25T20:10:02.594 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:02.594 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:02.594 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:02.594 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:02.594 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:02.594 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 139 2024-04-25T20:10:02.594 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.594 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:02.594 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.594 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:02.595 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:10:02.595 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.595 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.595 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.595 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:02.595 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:10:02.595 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.595 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.595 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.595 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:02.595 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121380+0000", 2024-04-25T20:10:02.595 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:02.595 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.595 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.595 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:02.595 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121385+0000", 2024-04-25T20:10:02.596 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.092e-06 2024-04-25T20:10:02.596 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.596 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.596 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:02.596 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121390+0000", 2024-04-25T20:10:02.596 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.5990000000000001e-06 2024-04-25T20:10:02.596 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.596 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.596 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:02.596 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121397+0000", 2024-04-25T20:10:02.596 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.6279999999999999e-06 2024-04-25T20:10:02.596 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.596 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.596 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:02.596 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125737+0000", 2024-04-25T20:10:02.597 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0043406010000000004 2024-04-25T20:10:02.597 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.597 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.597 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:02.597 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125781+0000", 2024-04-25T20:10:02.597 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.3958999999999997e-05 2024-04-25T20:10:02.597 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.597 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.597 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:02.597 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125849+0000", 2024-04-25T20:10:02.597 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8071999999999995e-05 2024-04-25T20:10:02.597 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.597 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.597 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:02.597 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.130908+0000", 2024-04-25T20:10:02.598 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0050586509999999999 2024-04-25T20:10:02.598 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.598 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.598 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:02.598 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132334+0000", 2024-04-25T20:10:02.598 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00142621 2024-04-25T20:10:02.598 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.598 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.598 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:02.598 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132369+0000", 2024-04-25T20:10:02.598 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4452000000000002e-05 2024-04-25T20:10:02.598 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.598 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.598 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:02.599 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132397+0000", 2024-04-25T20:10:02.599 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8065e-05 2024-04-25T20:10:02.599 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.599 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:02.599 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.599 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.599 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.599 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:02.599 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:02.599 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 44.398063295999997, 2024-04-25T20:10:02.599 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011069443, 2024-04-25T20:10:02.599 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:02.599 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:02.599 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:02.599 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:02.600 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:02.600 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 140 2024-04-25T20:10:02.600 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.600 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:02.600 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.600 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:02.600 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:02.600 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.600 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.600 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.600 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:02.600 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:02.600 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.600 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.601 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.601 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:02.601 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:02.601 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:02.601 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.601 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.601 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:02.601 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121456+0000", 2024-04-25T20:10:02.601 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.4469000000000003e-05 2024-04-25T20:10:02.601 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.601 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.601 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:02.601 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121462+0000", 2024-04-25T20:10:02.601 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.1890000000000004e-06 2024-04-25T20:10:02.601 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.602 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.602 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:02.602 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121466+0000", 2024-04-25T20:10:02.602 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3909999999999998e-06 2024-04-25T20:10:02.602 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.602 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.602 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:02.602 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126071+0000", 2024-04-25T20:10:02.602 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0046048979999999996 2024-04-25T20:10:02.602 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.602 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.602 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:02.602 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126141+0000", 2024-04-25T20:10:02.602 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0621e-05 2024-04-25T20:10:02.603 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.603 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.603 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:02.603 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126260+0000", 2024-04-25T20:10:02.603 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000118621 2024-04-25T20:10:02.603 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.603 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.603 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:02.603 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.131270+0000", 2024-04-25T20:10:02.603 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0050096949999999998 2024-04-25T20:10:02.603 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.603 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.603 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:02.603 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132403+0000", 2024-04-25T20:10:02.603 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011332779999999999 2024-04-25T20:10:02.604 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.604 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.604 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:02.604 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132443+0000", 2024-04-25T20:10:02.604 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0175000000000001e-05 2024-04-25T20:10:02.604 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.604 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.604 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:02.604 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132471+0000", 2024-04-25T20:10:02.604 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8399e-05 2024-04-25T20:10:02.604 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.604 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:02.604 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.604 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.604 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.605 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:02.605 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:10:02.605 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 44.397994511, 2024-04-25T20:10:02.605 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011071002, 2024-04-25T20:10:02.605 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:02.605 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:02.605 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:02.605 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:02.605 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:02.605 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 141 2024-04-25T20:10:02.605 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.605 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:02.605 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.605 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:02.605 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:10:02.606 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.606 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.606 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.606 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:02.606 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:10:02.606 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.606 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.606 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.606 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:02.606 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121470+0000", 2024-04-25T20:10:02.606 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:02.606 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.606 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.606 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:02.606 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121473+0000", 2024-04-25T20:10:02.607 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.01e-06 2024-04-25T20:10:02.607 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.607 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.607 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:02.607 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121478+0000", 2024-04-25T20:10:02.607 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.515e-06 2024-04-25T20:10:02.607 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.607 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.607 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:02.607 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121480+0000", 2024-04-25T20:10:02.607 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5749999999999999e-06 2024-04-25T20:10:02.607 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.607 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.607 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:02.607 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126813+0000", 2024-04-25T20:10:02.608 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0053320399999999997 2024-04-25T20:10:02.608 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.608 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.608 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:02.608 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126854+0000", 2024-04-25T20:10:02.608 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1582999999999998e-05 2024-04-25T20:10:02.608 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.608 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.608 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:02.608 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126923+0000", 2024-04-25T20:10:02.608 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8544e-05 2024-04-25T20:10:02.608 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.608 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.608 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:02.608 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.131573+0000", 2024-04-25T20:10:02.608 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0046504659999999998 2024-04-25T20:10:02.609 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.609 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.609 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:02.609 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132478+0000", 2024-04-25T20:10:02.609 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00090442400000000003 2024-04-25T20:10:02.609 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.609 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.609 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:02.609 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132513+0000", 2024-04-25T20:10:02.609 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5204999999999999e-05 2024-04-25T20:10:02.609 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.609 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.609 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:02.609 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132542+0000", 2024-04-25T20:10:02.609 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.904e-05 2024-04-25T20:10:02.610 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.610 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:02.610 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.610 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.610 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.610 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:156 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:02.610 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:02.610 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 43.900170215999999, 2024-04-25T20:10:02.610 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090072280000000008, 2024-04-25T20:10:02.610 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:02.610 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:02.610 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:02.610 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:02.610 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:02.610 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 156 2024-04-25T20:10:02.611 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.611 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:02.611 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.611 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:02.611 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:02.611 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.611 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.611 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.611 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:02.611 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:02.611 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.611 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.611 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.611 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:02.612 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:02.612 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:02.612 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.612 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.612 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:02.612 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619297+0000", 2024-04-25T20:10:02.612 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2979999999999999e-06 2024-04-25T20:10:02.612 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.612 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.612 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:02.612 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619300+0000", 2024-04-25T20:10:02.612 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.371e-06 2024-04-25T20:10:02.612 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.612 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.612 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:02.612 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619304+0000", 2024-04-25T20:10:02.613 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.304e-06 2024-04-25T20:10:02.613 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.613 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.613 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:02.613 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.622910+0000", 2024-04-25T20:10:02.613 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0036065049999999999 2024-04-25T20:10:02.613 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.613 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.613 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:02.613 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.622990+0000", 2024-04-25T20:10:02.613 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.9722999999999995e-05 2024-04-25T20:10:02.613 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.613 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.613 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:02.613 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623128+0000", 2024-04-25T20:10:02.614 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00013818399999999999 2024-04-25T20:10:02.614 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.614 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.614 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:02.614 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625155+0000", 2024-04-25T20:10:02.614 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0020267979999999998 2024-04-25T20:10:02.614 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.614 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.614 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:02.614 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628225+0000", 2024-04-25T20:10:02.614 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0030705680000000001 2024-04-25T20:10:02.614 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.614 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.614 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:02.615 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628268+0000", 2024-04-25T20:10:02.615 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.2404000000000003e-05 2024-04-25T20:10:02.615 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.615 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.615 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:02.615 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628302+0000", 2024-04-25T20:10:02.615 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4439e-05 2024-04-25T20:10:02.615 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.615 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:02.615 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.615 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.615 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.615 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:157 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:02.615 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:02.615 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 43.900158116999997, 2024-04-25T20:10:02.616 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090534889999999996, 2024-04-25T20:10:02.616 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:02.616 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:02.616 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:02.616 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:02.616 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:02.616 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 157 2024-04-25T20:10:02.616 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.616 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:02.616 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.616 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:02.616 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:02.616 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.616 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.616 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.617 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:02.617 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:02.617 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.617 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.617 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.617 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:02.617 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:02.617 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:02.617 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.617 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.617 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:02.617 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619309+0000", 2024-04-25T20:10:02.617 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.841e-06 2024-04-25T20:10:02.617 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.617 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.617 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:02.618 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619312+0000", 2024-04-25T20:10:02.618 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.084e-06 2024-04-25T20:10:02.618 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.618 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.618 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:02.618 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619314+0000", 2024-04-25T20:10:02.618 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.3860000000000001e-06 2024-04-25T20:10:02.618 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.618 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.618 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:02.618 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623718+0000", 2024-04-25T20:10:02.618 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0044038050000000002 2024-04-25T20:10:02.618 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.618 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.619 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:02.619 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623791+0000", 2024-04-25T20:10:02.619 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3181000000000003e-05 2024-04-25T20:10:02.619 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.619 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.619 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:02.619 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623922+0000", 2024-04-25T20:10:02.619 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00013117000000000001 2024-04-25T20:10:02.619 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.619 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.619 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:02.619 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625187+0000", 2024-04-25T20:10:02.619 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001264961 2024-04-25T20:10:02.619 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.619 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.619 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:02.620 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628308+0000", 2024-04-25T20:10:02.620 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.003120739 2024-04-25T20:10:02.620 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.620 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.620 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:02.620 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628331+0000", 2024-04-25T20:10:02.620 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2802999999999998e-05 2024-04-25T20:10:02.620 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.620 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.620 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:02.620 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628361+0000", 2024-04-25T20:10:02.620 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9760999999999998e-05 2024-04-25T20:10:02.620 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.620 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:02.620 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.621 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.621 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.621 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:158 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:02.621 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:10:02.621 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 43.900147670000003, 2024-04-25T20:10:02.621 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0091076409999999997, 2024-04-25T20:10:02.621 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:02.621 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:02.621 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:02.621 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:02.621 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:02.621 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 158 2024-04-25T20:10:02.621 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.621 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:02.622 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.622 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:02.622 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:10:02.622 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.622 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.622 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.622 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:02.622 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:10:02.622 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.622 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.622 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.622 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:02.622 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619317+0000", 2024-04-25T20:10:02.622 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967296 2024-04-25T20:10:02.622 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.623 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.623 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:02.623 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619364+0000", 2024-04-25T20:10:02.623 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.6572000000000001e-05 2024-04-25T20:10:02.623 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.623 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.623 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:02.623 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619371+0000", 2024-04-25T20:10:02.623 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3860000000000001e-06 2024-04-25T20:10:02.623 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.623 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.623 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:02.623 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619382+0000", 2024-04-25T20:10:02.623 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0173999999999999e-05 2024-04-25T20:10:02.623 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.624 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.624 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:02.624 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624216+0000", 2024-04-25T20:10:02.624 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048340350000000004 2024-04-25T20:10:02.624 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.624 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.624 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:02.624 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624260+0000", 2024-04-25T20:10:02.624 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4558999999999998e-05 2024-04-25T20:10:02.624 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.624 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.624 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:02.624 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624333+0000", 2024-04-25T20:10:02.624 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2744999999999996e-05 2024-04-25T20:10:02.624 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.625 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.625 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:02.625 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625465+0000", 2024-04-25T20:10:02.625 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001132454 2024-04-25T20:10:02.625 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.625 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.625 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:02.625 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628365+0000", 2024-04-25T20:10:02.625 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0028994950000000002 2024-04-25T20:10:02.625 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.625 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.625 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:02.625 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628386+0000", 2024-04-25T20:10:02.625 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.1367000000000001e-05 2024-04-25T20:10:02.625 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.626 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.626 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:02.626 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628425+0000", 2024-04-25T20:10:02.626 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.9087e-05 2024-04-25T20:10:02.626 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.626 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:02.626 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.626 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.626 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.626 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:159 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:02.626 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:02.626 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 43.900079017000003, 2024-04-25T20:10:02.626 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090948999999999995, 2024-04-25T20:10:02.626 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:02.627 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:02.627 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:02.627 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:02.627 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:02.627 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 159 2024-04-25T20:10:02.627 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.627 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:02.627 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.627 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:02.627 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:02.627 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.627 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.627 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.627 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:02.627 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:02.627 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:02.628 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.628 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.628 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:02.628 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:02.628 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:02.628 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.628 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.628 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:02.628 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619389+0000", 2024-04-25T20:10:02.628 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.734e-06 2024-04-25T20:10:02.628 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.628 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.628 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:02.628 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619392+0000", 2024-04-25T20:10:02.628 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6219999999999998e-06 2024-04-25T20:10:02.629 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.629 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.629 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:02.629 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619403+0000", 2024-04-25T20:10:02.629 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0987e-05 2024-04-25T20:10:02.629 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.629 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.629 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:02.629 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624667+0000", 2024-04-25T20:10:02.629 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0052633970000000004 2024-04-25T20:10:02.629 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.629 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.629 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:02.629 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624741+0000", 2024-04-25T20:10:02.629 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.4583000000000001e-05 2024-04-25T20:10:02.630 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.630 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.630 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:02.630 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624857+0000", 2024-04-25T20:10:02.630 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000116118 2024-04-25T20:10:02.630 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.630 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.630 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:02.630 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.626013+0000", 2024-04-25T20:10:02.630 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001155473 2024-04-25T20:10:02.630 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.630 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.630 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:02.630 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628432+0000", 2024-04-25T20:10:02.631 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0024189379999999998 2024-04-25T20:10:02.631 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.631 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.631 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:02.631 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628454+0000", 2024-04-25T20:10:02.631 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2059999999999999e-05 2024-04-25T20:10:02.631 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:02.631 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:02.631 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:02.631 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628481+0000", 2024-04-25T20:10:02.631 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.7300999999999998e-05 2024-04-25T20:10:02.631 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.631 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:02.631 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.631 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:02.632 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:02.632 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:02.633 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T20:10:02.634 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:02.632+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:02.734 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:02.733+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:02.742 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:02.742 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:02.742 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:02.742 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:02.751 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T20:10:02.766 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T20:10:02.838 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:02.838+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:02.861 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:02.861 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:02.861 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:02.861 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:02.861 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:02.871 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T20:10:02.929 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:10:02.929 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:10:02.935 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:02.935+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:02.950 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:10:02.950 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:10:02.970 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:10:02.970 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:10:02.988 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:10:02.988 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:10:03.008 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:10:03.008 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:10:03.025 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:03.025 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:03.026 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:10:03.026 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:10:03.035 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T20:10:03.036 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:03.036+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:03.044 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:10:03.044 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:10:03.063 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:10:03.063 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:10:03.138 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:03.138+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:03.164 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:03.164 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:03.174 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T20:10:03.238 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:03.238+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:03.283 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:03.283 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:03.292 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T20:10:03.338 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is set 2024-04-25T20:10:03.339 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:03.339+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:03.440 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:03.440+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:03.442 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:03.442 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:03.451 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T20:10:03.541 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:03.541+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:03.599 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:03.599 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:03.608 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T20:10:03.652 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:03.652+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:03.714 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:03.714 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:03.723 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T20:10:03.764 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:03.764+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:03.844 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:03.844+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:03.874 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:03.874 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:03.883 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T20:10:03.967 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:03.967+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:04.034 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:04.034 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:04.044 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T20:10:04.046 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:04.046+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:04.146 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:04.146+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:04.190 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:04.190 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:04.200 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T20:10:04.248 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:04.248+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:04.349 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:04.349+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:04.354 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:04.354 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:04.364 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T20:10:04.449 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:04.449+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:04.478 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:04.478 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:04.487 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T20:10:04.550 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:04.549+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:04.589 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:04.589 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:04.598 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T20:10:04.651 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:04.651+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:04.739 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:04.739 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:04.749 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T20:10:04.752 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:04.751+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:04.850 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:04.850 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:04.853 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:04.852+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:04.859 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T20:10:04.954 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:04.954+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:04.960 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:04.960 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:04.969 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T20:10:05.055 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:05.054+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:05.070 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:05.070 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:05.076 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T20:10:05.079 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T20:10:05.156 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:05.156+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:05.202 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:10:05.202 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:10:05.219 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:10:05.219 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:10:05.222 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:05.222 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:05.231 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T20:10:05.240 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:10:05.240 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:10:05.256 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:10:05.257 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:10:05.257 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:05.256+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:05.273 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:10:05.273 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:10:05.290 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:10:05.290 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:10:05.306 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:10:05.306 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:10:05.323 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:10:05.323 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:10:05.334 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:05.334 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:05.334 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:05.334 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:05.334 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:05.343 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T20:10:05.353 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-25T20:10:05.375 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:05.375+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:05.444 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:05.444 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:05.444 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:05.444 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:10:05.444 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:05.453 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T20:10:05.458 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:05.458+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:05.555 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:05.555 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:05.555 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:10:05.556 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:05.559 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:05.559+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:05.564 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T20:10:05.660 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:05.660+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:05.665 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:05.666 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:05.666 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:05.666 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:05.666 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:05.674 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T20:10:05.761 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:05.761+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:05.818 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:05.818 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:05.819 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:05.819 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [ 2024-04-25T20:10:05.819 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.819 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:05.819 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:10:05.819 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 49.034697080999997, 2024-04-25T20:10:05.819 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.34154860100000001, 2024-04-25T20:10:05.819 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:05.819 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:05.819 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:05.819 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:05.819 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:05.819 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 1 2024-04-25T20:10:05.819 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.820 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:05.820 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.820 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:05.820 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:10:05.820 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.820 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.820 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.820 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:05.820 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:10:05.820 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.820 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.820 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.820 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:05.820 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782261+0000", 2024-04-25T20:10:05.821 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999986 2024-04-25T20:10:05.821 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.821 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.821 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:05.821 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782268+0000", 2024-04-25T20:10:05.821 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.8140000000000001e-06 2024-04-25T20:10:05.821 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.821 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.821 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:05.821 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782279+0000", 2024-04-25T20:10:05.821 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0665999999999999e-05 2024-04-25T20:10:05.821 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.821 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.821 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:05.821 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782290+0000", 2024-04-25T20:10:05.821 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.1343e-05 2024-04-25T20:10:05.822 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.822 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.822 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:05.822 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782340+0000", 2024-04-25T20:10:05.822 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.9311999999999998e-05 2024-04-25T20:10:05.822 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.822 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.822 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for peered", 2024-04-25T20:10:05.822 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782348+0000", 2024-04-25T20:10:05.822 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.0690000000000002e-06 2024-04-25T20:10:05.822 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.822 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.822 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:05.822 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120520+0000", 2024-04-25T20:10:05.823 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.33817188100000001 2024-04-25T20:10:05.823 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.823 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.823 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:05.823 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120656+0000", 2024-04-25T20:10:05.823 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000135897 2024-04-25T20:10:05.823 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.823 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.823 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:05.823 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120802+0000", 2024-04-25T20:10:05.823 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00014691199999999999 2024-04-25T20:10:05.823 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.823 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.823 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:05.823 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.122368+0000", 2024-04-25T20:10:05.824 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0015651040000000001 2024-04-25T20:10:05.824 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.824 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.824 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:05.824 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123729+0000", 2024-04-25T20:10:05.824 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0013610499999999999 2024-04-25T20:10:05.824 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.824 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.824 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:05.824 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123769+0000", 2024-04-25T20:10:05.824 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0719000000000001e-05 2024-04-25T20:10:05.824 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.824 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.824 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:05.825 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123811+0000", 2024-04-25T20:10:05.825 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1267999999999999e-05 2024-04-25T20:10:05.825 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.825 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:05.825 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.825 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.825 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.825 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:67 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:05.825 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:10:05.825 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 48.605278194, 2024-04-25T20:10:05.825 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.006947651, 2024-04-25T20:10:05.825 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:05.825 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:05.825 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:05.825 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:05.826 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:05.826 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 67 2024-04-25T20:10:05.826 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.826 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:05.826 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.826 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:05.826 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:10:05.826 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.826 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.826 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.826 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:05.826 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:10:05.826 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.826 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.827 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.827 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:05.827 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211680+0000", 2024-04-25T20:10:05.827 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:05.827 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.827 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.827 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:05.827 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211684+0000", 2024-04-25T20:10:05.827 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4249999999999998e-06 2024-04-25T20:10:05.827 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.827 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.827 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:05.827 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211690+0000", 2024-04-25T20:10:05.827 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.7360000000000004e-06 2024-04-25T20:10:05.827 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.828 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.828 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:05.828 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211694+0000", 2024-04-25T20:10:05.828 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4860000000000002e-06 2024-04-25T20:10:05.828 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.828 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.828 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:05.828 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211727+0000", 2024-04-25T20:10:05.828 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3445999999999998e-05 2024-04-25T20:10:05.828 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.829 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.829 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:05.829 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211778+0000", 2024-04-25T20:10:05.829 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1050000000000001e-05 2024-04-25T20:10:05.829 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.829 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.829 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:05.829 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211851+0000", 2024-04-25T20:10:05.829 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2914e-05 2024-04-25T20:10:05.829 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.829 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.829 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:05.829 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213824+0000", 2024-04-25T20:10:05.829 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0019726520000000001 2024-04-25T20:10:05.829 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.829 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.830 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:05.830 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218579+0000", 2024-04-25T20:10:05.830 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0047553159999999999 2024-04-25T20:10:05.830 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.830 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.830 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:05.830 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218607+0000", 2024-04-25T20:10:05.830 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8144999999999999e-05 2024-04-25T20:10:05.830 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.830 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.830 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:05.830 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218629+0000", 2024-04-25T20:10:05.830 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.1044e-05 2024-04-25T20:10:05.830 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.831 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:05.831 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.831 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.831 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.831 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:68 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:05.831 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:05.831 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 48.605259853, 2024-04-25T20:10:05.831 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012255596000000001, 2024-04-25T20:10:05.831 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:05.831 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:05.831 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:05.831 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:05.831 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:05.831 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 68 2024-04-25T20:10:05.831 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.832 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:05.832 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.832 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:05.832 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:05.832 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.832 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.832 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.832 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:05.832 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:05.832 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.832 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.832 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.832 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:05.832 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:05.832 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:05.833 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.833 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.833 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:05.833 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211701+0000", 2024-04-25T20:10:05.833 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2029999999999999e-06 2024-04-25T20:10:05.833 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.833 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.833 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:05.833 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211705+0000", 2024-04-25T20:10:05.833 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.2320000000000003e-06 2024-04-25T20:10:05.833 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.833 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.833 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:05.833 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211708+0000", 2024-04-25T20:10:05.833 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.6290000000000002e-06 2024-04-25T20:10:05.834 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.834 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.834 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:05.834 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212136+0000", 2024-04-25T20:10:05.834 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00042764099999999998 2024-04-25T20:10:05.834 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.834 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.834 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:05.834 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212212+0000", 2024-04-25T20:10:05.834 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.6315000000000004e-05 2024-04-25T20:10:05.834 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.834 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.834 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:05.834 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212306+0000", 2024-04-25T20:10:05.835 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.3556999999999999e-05 2024-04-25T20:10:05.835 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.835 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.835 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:05.835 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213829+0000", 2024-04-25T20:10:05.835 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001523256 2024-04-25T20:10:05.835 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.835 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.835 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:05.835 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223904+0000", 2024-04-25T20:10:05.835 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010075387 2024-04-25T20:10:05.835 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.835 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.835 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:05.835 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223934+0000", 2024-04-25T20:10:05.836 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9988000000000002e-05 2024-04-25T20:10:05.836 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.836 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.836 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:05.836 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223955+0000", 2024-04-25T20:10:05.836 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0630999999999999e-05 2024-04-25T20:10:05.836 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.836 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:05.836 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.836 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.836 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.836 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:69 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:05.836 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:05.836 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 48.605238118000003, 2024-04-25T20:10:05.836 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012316863000000001, 2024-04-25T20:10:05.837 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:05.837 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:05.837 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:05.837 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:05.837 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:05.837 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 69 2024-04-25T20:10:05.837 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.837 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:05.837 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.837 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:05.837 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:05.837 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.837 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.837 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.837 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:05.838 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:05.838 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.838 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.838 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.838 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:05.838 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:05.838 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:05.838 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.838 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.838 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:05.838 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211762+0000", 2024-04-25T20:10:05.838 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1703999999999999e-05 2024-04-25T20:10:05.838 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.838 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.839 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:05.839 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211768+0000", 2024-04-25T20:10:05.839 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.3430000000000003e-06 2024-04-25T20:10:05.839 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.839 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.839 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:05.839 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211771+0000", 2024-04-25T20:10:05.839 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.18e-06 2024-04-25T20:10:05.839 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.839 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.839 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:05.839 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212652+0000", 2024-04-25T20:10:05.839 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00088095800000000004 2024-04-25T20:10:05.839 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.839 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.840 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:05.840 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212703+0000", 2024-04-25T20:10:05.840 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.0655999999999997e-05 2024-04-25T20:10:05.840 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.840 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.840 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:05.840 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212781+0000", 2024-04-25T20:10:05.840 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.8852999999999999e-05 2024-04-25T20:10:05.840 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.840 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.840 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:05.840 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.214131+0000", 2024-04-25T20:10:05.840 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0013496039999999999 2024-04-25T20:10:05.840 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.840 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.841 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:05.841 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223986+0000", 2024-04-25T20:10:05.841 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.009854804 2024-04-25T20:10:05.841 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.841 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.841 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:05.841 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.224017+0000", 2024-04-25T20:10:05.841 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.1068999999999997e-05 2024-04-25T20:10:05.841 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.841 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.841 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:05.841 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.224038+0000", 2024-04-25T20:10:05.841 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0965e-05 2024-04-25T20:10:05.841 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.841 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:05.842 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.842 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.842 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.842 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:70 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:05.842 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:05.842 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 48.605183037000003, 2024-04-25T20:10:05.842 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.015776348999999999, 2024-04-25T20:10:05.842 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:05.842 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:05.842 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:05.842 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:05.842 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:05.842 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 70 2024-04-25T20:10:05.842 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.843 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:05.843 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.843 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:05.843 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:05.843 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.843 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.843 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.843 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:05.843 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:05.843 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.843 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.843 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.843 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:05.843 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:05.843 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:05.844 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.844 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.844 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:05.844 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211778+0000", 2024-04-25T20:10:05.844 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5330000000000001e-06 2024-04-25T20:10:05.844 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.844 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.844 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:05.844 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211782+0000", 2024-04-25T20:10:05.844 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4079999999999998e-06 2024-04-25T20:10:05.844 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.844 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.844 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:05.844 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211786+0000", 2024-04-25T20:10:05.844 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.6190000000000002e-06 2024-04-25T20:10:05.845 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.845 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.845 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:05.845 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213374+0000", 2024-04-25T20:10:05.845 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0015881859999999999 2024-04-25T20:10:05.845 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.845 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.845 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:05.845 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213478+0000", 2024-04-25T20:10:05.845 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000103464 2024-04-25T20:10:05.845 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.845 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.845 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:05.845 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213601+0000", 2024-04-25T20:10:05.845 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00012324300000000001 2024-04-25T20:10:05.846 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.846 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.846 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:05.846 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.214788+0000", 2024-04-25T20:10:05.846 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011868569999999999 2024-04-25T20:10:05.846 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.846 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.846 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:05.846 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227504+0000", 2024-04-25T20:10:05.846 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012715749 2024-04-25T20:10:05.846 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.846 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.846 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:05.846 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227532+0000", 2024-04-25T20:10:05.847 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.7937e-05 2024-04-25T20:10:05.847 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.847 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.847 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:05.847 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227552+0000", 2024-04-25T20:10:05.847 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0772999999999999e-05 2024-04-25T20:10:05.847 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.847 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:05.847 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.847 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.847 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.847 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:131 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:05.847 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:10:05.847 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 47.696173192000003, 2024-04-25T20:10:05.847 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0086158810000000006, 2024-04-25T20:10:05.848 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:05.848 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:05.848 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:05.848 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:05.848 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:05.848 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 131 2024-04-25T20:10:05.848 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.848 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:05.848 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.848 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:05.848 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:10:05.848 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.848 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.848 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.849 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:05.849 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:10:05.849 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.849 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.849 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.849 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:05.849 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120785+0000", 2024-04-25T20:10:05.849 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:05.849 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.849 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.849 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:05.849 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120880+0000", 2024-04-25T20:10:05.849 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.4925000000000003e-05 2024-04-25T20:10:05.849 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.850 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.850 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:05.850 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120891+0000", 2024-04-25T20:10:05.850 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0915e-05 2024-04-25T20:10:05.850 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.850 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.850 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:05.850 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120895+0000", 2024-04-25T20:10:05.850 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.8070000000000001e-06 2024-04-25T20:10:05.850 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.850 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.850 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:05.850 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121640+0000", 2024-04-25T20:10:05.850 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00074468500000000003 2024-04-25T20:10:05.851 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.851 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.851 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:05.851 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121684+0000", 2024-04-25T20:10:05.851 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4452000000000001e-05 2024-04-25T20:10:05.851 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.851 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.851 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:05.851 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121753+0000", 2024-04-25T20:10:05.851 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8508000000000002e-05 2024-04-25T20:10:05.851 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.851 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.851 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:05.851 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123618+0000", 2024-04-25T20:10:05.852 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0018655099999999999 2024-04-25T20:10:05.852 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.852 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.852 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:05.852 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129248+0000", 2024-04-25T20:10:05.852 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0056295540000000002 2024-04-25T20:10:05.852 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.852 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.852 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:05.852 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129368+0000", 2024-04-25T20:10:05.852 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000120244 2024-04-25T20:10:05.852 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.852 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.852 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:05.853 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129402+0000", 2024-04-25T20:10:05.853 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3812000000000001e-05 2024-04-25T20:10:05.853 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.853 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:05.853 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.853 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.853 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.853 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:132 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:05.853 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:05.853 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 47.696058981999997, 2024-04-25T20:10:05.853 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0085898169999999996, 2024-04-25T20:10:05.853 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:05.853 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:05.853 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:05.853 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:05.854 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:05.854 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 132 2024-04-25T20:10:05.854 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.854 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:05.854 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.854 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:05.854 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:05.854 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.854 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.854 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.854 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:05.854 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:05.854 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.854 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.855 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.855 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:05.855 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:05.855 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:05.855 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.855 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.855 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:05.855 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120902+0000", 2024-04-25T20:10:05.855 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.6129999999999998e-06 2024-04-25T20:10:05.855 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.855 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.855 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:05.855 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120906+0000", 2024-04-25T20:10:05.855 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6569999999999999e-06 2024-04-25T20:10:05.856 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.856 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.856 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:05.856 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120909+0000", 2024-04-25T20:10:05.856 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5550000000000001e-06 2024-04-25T20:10:05.856 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.856 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.856 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:05.856 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122098+0000", 2024-04-25T20:10:05.856 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011898049999999999 2024-04-25T20:10:05.856 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.856 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.856 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:05.856 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122172+0000", 2024-04-25T20:10:05.856 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3917000000000004e-05 2024-04-25T20:10:05.857 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.857 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.857 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:05.857 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122288+0000", 2024-04-25T20:10:05.857 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00011566 2024-04-25T20:10:05.857 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.857 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.857 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:05.857 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124519+0000", 2024-04-25T20:10:05.857 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0022308359999999999 2024-04-25T20:10:05.857 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.857 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.857 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:05.858 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129414+0000", 2024-04-25T20:10:05.858 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048953720000000003 2024-04-25T20:10:05.858 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.858 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.858 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:05.858 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129458+0000", 2024-04-25T20:10:05.858 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.3426000000000001e-05 2024-04-25T20:10:05.858 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.858 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.858 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:05.858 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129490+0000", 2024-04-25T20:10:05.858 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2329999999999997e-05 2024-04-25T20:10:05.858 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.859 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:05.859 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.859 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.859 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.859 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:133 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:05.859 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:05.859 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 47.696005112999998, 2024-04-25T20:10:05.859 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0086066270000000004, 2024-04-25T20:10:05.859 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:05.859 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:05.859 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:05.859 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:05.860 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:05.860 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 133 2024-04-25T20:10:05.860 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.860 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:05.860 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.860 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:05.860 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:05.860 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.860 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.860 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.860 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:05.860 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:05.860 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.860 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.860 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.860 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:05.861 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:05.861 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:05.861 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.861 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.861 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:05.861 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120956+0000", 2024-04-25T20:10:05.861 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5289999999999998e-06 2024-04-25T20:10:05.861 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.861 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.861 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:05.861 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120962+0000", 2024-04-25T20:10:05.861 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.9490000000000001e-06 2024-04-25T20:10:05.861 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.861 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.861 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:05.861 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120965+0000", 2024-04-25T20:10:05.862 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.1860000000000001e-06 2024-04-25T20:10:05.862 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.862 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.862 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:05.862 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122666+0000", 2024-04-25T20:10:05.862 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0017004500000000001 2024-04-25T20:10:05.862 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.862 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.862 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:05.862 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122741+0000", 2024-04-25T20:10:05.862 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.5297999999999996e-05 2024-04-25T20:10:05.862 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.862 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.862 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:05.862 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122858+0000", 2024-04-25T20:10:05.863 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00011723899999999999 2024-04-25T20:10:05.863 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.863 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.863 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:05.863 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124625+0000", 2024-04-25T20:10:05.863 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0017670100000000001 2024-04-25T20:10:05.863 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.863 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.863 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:05.863 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129497+0000", 2024-04-25T20:10:05.863 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048713580000000001 2024-04-25T20:10:05.863 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.863 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.863 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:05.863 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129532+0000", 2024-04-25T20:10:05.863 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5704999999999998e-05 2024-04-25T20:10:05.864 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.864 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.864 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:05.864 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129561+0000", 2024-04-25T20:10:05.864 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8283e-05 2024-04-25T20:10:05.864 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.864 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:05.864 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.864 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.864 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.864 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:134 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:05.864 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:05.864 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 47.695902914999998, 2024-04-25T20:10:05.864 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0085771179999999999, 2024-04-25T20:10:05.864 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:05.864 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:05.865 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:05.865 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:05.865 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:05.865 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 134 2024-04-25T20:10:05.865 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.865 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:05.865 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.865 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:05.865 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:05.865 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.865 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.865 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.865 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:05.865 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:05.865 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.866 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.866 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.866 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:05.866 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:05.866 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:05.866 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.866 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.866 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:05.866 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121157+0000", 2024-04-25T20:10:05.866 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00010104599999999999 2024-04-25T20:10:05.866 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.866 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.866 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:05.867 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121189+0000", 2024-04-25T20:10:05.867 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2589000000000003e-05 2024-04-25T20:10:05.867 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.867 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.867 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:05.867 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121195+0000", 2024-04-25T20:10:05.867 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.083e-06 2024-04-25T20:10:05.867 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.867 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.867 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:05.867 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123136+0000", 2024-04-25T20:10:05.867 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0019409780000000001 2024-04-25T20:10:05.867 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.867 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.867 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:05.868 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123181+0000", 2024-04-25T20:10:05.868 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4267999999999997e-05 2024-04-25T20:10:05.868 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.868 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.868 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:05.868 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123247+0000", 2024-04-25T20:10:05.868 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.6086000000000004e-05 2024-04-25T20:10:05.868 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.868 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.868 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:05.868 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125639+0000", 2024-04-25T20:10:05.868 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.002392 2024-04-25T20:10:05.868 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.868 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.869 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:05.869 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129567+0000", 2024-04-25T20:10:05.869 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0039282170000000003 2024-04-25T20:10:05.869 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.869 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.869 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:05.869 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129605+0000", 2024-04-25T20:10:05.869 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.7660999999999997e-05 2024-04-25T20:10:05.869 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.869 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.869 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:05.869 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129633+0000", 2024-04-25T20:10:05.869 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8649999999999998e-05 2024-04-25T20:10:05.869 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.869 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:05.870 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.870 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.870 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.870 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:135 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:05.870 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:10:05.870 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 47.695743909000001, 2024-04-25T20:10:05.870 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010898453000000001, 2024-04-25T20:10:05.870 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:05.870 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:05.870 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:05.870 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:05.870 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:05.870 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 135 2024-04-25T20:10:05.871 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.871 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:05.871 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.871 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:05.871 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:10:05.871 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.871 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.871 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.871 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:05.871 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:10:05.871 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.871 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.871 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.871 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:05.872 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121214+0000", 2024-04-25T20:10:05.872 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:05.872 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.872 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.872 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:05.872 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121218+0000", 2024-04-25T20:10:05.872 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6619999999999998e-06 2024-04-25T20:10:05.872 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.872 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.872 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:05.872 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121223+0000", 2024-04-25T20:10:05.872 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1070000000000004e-06 2024-04-25T20:10:05.872 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.872 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.873 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:05.873 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121227+0000", 2024-04-25T20:10:05.873 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2820000000000001e-06 2024-04-25T20:10:05.873 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.873 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.873 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:05.873 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123649+0000", 2024-04-25T20:10:05.873 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0024221970000000001 2024-04-25T20:10:05.873 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.873 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.873 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:05.873 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123723+0000", 2024-04-25T20:10:05.873 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.4386000000000002e-05 2024-04-25T20:10:05.873 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.873 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.874 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:05.874 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123844+0000", 2024-04-25T20:10:05.874 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000121249 2024-04-25T20:10:05.874 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.874 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.874 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:05.874 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125670+0000", 2024-04-25T20:10:05.874 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001825901 2024-04-25T20:10:05.874 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.874 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.874 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:05.874 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132043+0000", 2024-04-25T20:10:05.874 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0063724539999999996 2024-04-25T20:10:05.874 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.875 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.875 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:05.875 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132084+0000", 2024-04-25T20:10:05.875 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0963000000000001e-05 2024-04-25T20:10:05.875 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.875 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.875 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:05.875 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132114+0000", 2024-04-25T20:10:05.875 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9927e-05 2024-04-25T20:10:05.875 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.875 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:05.875 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.875 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.875 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.876 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:136 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:05.876 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:10:05.876 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 47.695728383000002, 2024-04-25T20:10:05.876 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010952513000000001, 2024-04-25T20:10:05.876 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:05.876 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:05.876 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:05.876 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:05.876 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:05.876 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 136 2024-04-25T20:10:05.876 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.876 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:05.876 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.876 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:05.876 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:10:05.877 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.877 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.877 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.877 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:05.877 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:10:05.877 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.877 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.877 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.877 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:05.877 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121230+0000", 2024-04-25T20:10:05.877 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:05.877 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.877 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.877 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:05.877 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121236+0000", 2024-04-25T20:10:05.877 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.7320000000000003e-06 2024-04-25T20:10:05.877 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.878 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.878 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:05.878 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121242+0000", 2024-04-25T20:10:05.878 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.7810000000000002e-06 2024-04-25T20:10:05.878 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.878 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.878 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:05.878 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121246+0000", 2024-04-25T20:10:05.878 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.639e-06 2024-04-25T20:10:05.878 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.878 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.878 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:05.878 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124150+0000", 2024-04-25T20:10:05.878 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.002903816 2024-04-25T20:10:05.878 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.879 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.879 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:05.879 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124195+0000", 2024-04-25T20:10:05.879 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.5042e-05 2024-04-25T20:10:05.879 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.879 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.879 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:05.879 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124266+0000", 2024-04-25T20:10:05.879 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0234999999999994e-05 2024-04-25T20:10:05.879 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.879 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.879 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:05.879 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125711+0000", 2024-04-25T20:10:05.879 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0014458800000000001 2024-04-25T20:10:05.879 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.879 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.880 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:05.880 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132120+0000", 2024-04-25T20:10:05.880 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0064084399999999996 2024-04-25T20:10:05.880 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.880 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.880 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:05.880 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132155+0000", 2024-04-25T20:10:05.880 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4938000000000003e-05 2024-04-25T20:10:05.880 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.880 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.880 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:05.880 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132183+0000", 2024-04-25T20:10:05.880 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8360999999999998e-05 2024-04-25T20:10:05.880 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.880 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:05.880 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.880 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.881 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.881 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:05.881 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:05.881 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 47.695678651000001, 2024-04-25T20:10:05.881 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.01097684, 2024-04-25T20:10:05.881 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:05.881 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:05.881 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:05.881 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:05.881 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:05.881 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 137 2024-04-25T20:10:05.881 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.881 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:05.881 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.881 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:05.881 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:05.881 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.882 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.882 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.882 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:05.882 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:05.882 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.882 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.882 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.882 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:05.882 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:05.882 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:05.882 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.882 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.882 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:05.882 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121328+0000", 2024-04-25T20:10:05.882 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.7908999999999998e-05 2024-04-25T20:10:05.883 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.883 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.883 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:05.883 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121337+0000", 2024-04-25T20:10:05.883 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.9369999999999993e-06 2024-04-25T20:10:05.883 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.883 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.883 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:05.883 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121343+0000", 2024-04-25T20:10:05.883 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.8089999999999998e-06 2024-04-25T20:10:05.883 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.883 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.883 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:05.883 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124656+0000", 2024-04-25T20:10:05.883 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0033133839999999999 2024-04-25T20:10:05.884 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.884 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.884 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:05.884 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124730+0000", 2024-04-25T20:10:05.884 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3565000000000005e-05 2024-04-25T20:10:05.884 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.884 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.884 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:05.884 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124846+0000", 2024-04-25T20:10:05.884 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000116416 2024-04-25T20:10:05.884 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.884 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.884 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:05.884 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126756+0000", 2024-04-25T20:10:05.884 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001909991 2024-04-25T20:10:05.884 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.884 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.885 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:05.885 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132190+0000", 2024-04-25T20:10:05.885 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0054343880000000001 2024-04-25T20:10:05.885 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.885 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.885 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:05.885 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132227+0000", 2024-04-25T20:10:05.885 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6875000000000002e-05 2024-04-25T20:10:05.885 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.885 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.885 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:05.885 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132257+0000", 2024-04-25T20:10:05.885 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.0046e-05 2024-04-25T20:10:05.885 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.885 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:05.886 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.886 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.886 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.886 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:05.886 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:05.886 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 47.695608692999997, 2024-04-25T20:10:05.886 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010978653999999999, 2024-04-25T20:10:05.886 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:05.886 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:05.886 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:05.886 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:05.886 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:05.886 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 138 2024-04-25T20:10:05.886 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.887 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:05.887 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.887 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:05.887 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:05.887 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.887 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.887 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.887 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:05.887 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:05.887 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.887 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.887 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.887 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:05.887 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:05.887 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:05.888 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.888 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.888 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:05.888 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121355+0000", 2024-04-25T20:10:05.888 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.9869999999999999e-06 2024-04-25T20:10:05.888 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.888 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.888 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:05.888 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121361+0000", 2024-04-25T20:10:05.888 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.4620000000000001e-06 2024-04-25T20:10:05.888 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.888 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.888 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:05.888 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121373+0000", 2024-04-25T20:10:05.888 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.2035000000000001e-05 2024-04-25T20:10:05.889 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.889 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.889 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:05.889 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125267+0000", 2024-04-25T20:10:05.889 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0038939230000000001 2024-04-25T20:10:05.889 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.889 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.889 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:05.889 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125309+0000", 2024-04-25T20:10:05.889 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1690999999999998e-05 2024-04-25T20:10:05.889 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.889 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.889 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:05.889 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125385+0000", 2024-04-25T20:10:05.890 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.6162999999999996e-05 2024-04-25T20:10:05.890 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.890 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.890 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:05.890 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126798+0000", 2024-04-25T20:10:05.890 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0014124680000000001 2024-04-25T20:10:05.890 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.890 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.890 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:05.890 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132264+0000", 2024-04-25T20:10:05.890 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0054661689999999999 2024-04-25T20:10:05.890 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.890 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.890 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:05.891 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132299+0000", 2024-04-25T20:10:05.891 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5272999999999999e-05 2024-04-25T20:10:05.891 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.891 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.891 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:05.891 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132329+0000", 2024-04-25T20:10:05.891 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.0080999999999999e-05 2024-04-25T20:10:05.891 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.891 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:05.891 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.891 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.891 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.891 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:05.891 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:10:05.891 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 47.695578118, 2024-04-25T20:10:05.892 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011015853000000001, 2024-04-25T20:10:05.892 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:05.892 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:05.892 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:05.892 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:05.892 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:05.892 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 139 2024-04-25T20:10:05.892 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.892 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:05.892 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.892 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:05.892 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:10:05.892 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.892 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.892 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.893 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:05.893 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:10:05.893 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.893 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.893 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.893 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:05.893 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121380+0000", 2024-04-25T20:10:05.893 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:05.893 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.893 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.893 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:05.893 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121385+0000", 2024-04-25T20:10:05.893 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.092e-06 2024-04-25T20:10:05.893 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.893 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.893 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:05.893 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121390+0000", 2024-04-25T20:10:05.894 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.5990000000000001e-06 2024-04-25T20:10:05.894 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.894 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.894 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:05.894 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121397+0000", 2024-04-25T20:10:05.894 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.6279999999999999e-06 2024-04-25T20:10:05.894 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.894 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.894 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:05.894 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125737+0000", 2024-04-25T20:10:05.894 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0043406010000000004 2024-04-25T20:10:05.894 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.894 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.894 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:05.895 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125781+0000", 2024-04-25T20:10:05.895 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.3958999999999997e-05 2024-04-25T20:10:05.895 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.895 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.895 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:05.895 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125849+0000", 2024-04-25T20:10:05.895 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8071999999999995e-05 2024-04-25T20:10:05.895 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.895 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.895 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:05.895 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.130908+0000", 2024-04-25T20:10:05.895 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0050586509999999999 2024-04-25T20:10:05.895 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.895 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.895 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:05.895 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132334+0000", 2024-04-25T20:10:05.895 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00142621 2024-04-25T20:10:05.896 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.896 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.896 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:05.896 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132369+0000", 2024-04-25T20:10:05.896 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4452000000000002e-05 2024-04-25T20:10:05.896 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.896 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.896 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:05.896 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132397+0000", 2024-04-25T20:10:05.896 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8065e-05 2024-04-25T20:10:05.896 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.896 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:05.896 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.896 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.897 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.897 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:05.897 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:05.897 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 47.695557074, 2024-04-25T20:10:05.897 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011069443, 2024-04-25T20:10:05.897 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:05.897 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:05.897 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:05.897 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:05.897 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:05.897 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 140 2024-04-25T20:10:05.897 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.897 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:05.897 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.897 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:05.898 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:05.898 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.898 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.898 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.898 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:05.898 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:05.898 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.898 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.898 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.898 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:05.898 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:05.898 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:05.898 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.898 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.899 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:05.899 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121456+0000", 2024-04-25T20:10:05.899 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.4469000000000003e-05 2024-04-25T20:10:05.899 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.899 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.899 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:05.899 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121462+0000", 2024-04-25T20:10:05.899 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.1890000000000004e-06 2024-04-25T20:10:05.899 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.899 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.899 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:05.899 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121466+0000", 2024-04-25T20:10:05.899 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3909999999999998e-06 2024-04-25T20:10:05.899 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.900 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.900 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:05.900 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126071+0000", 2024-04-25T20:10:05.900 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0046048979999999996 2024-04-25T20:10:05.900 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.900 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.900 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:05.900 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126141+0000", 2024-04-25T20:10:05.900 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0621e-05 2024-04-25T20:10:05.900 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.900 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.900 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:05.900 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126260+0000", 2024-04-25T20:10:05.900 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000118621 2024-04-25T20:10:05.900 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.901 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.901 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:05.901 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.131270+0000", 2024-04-25T20:10:05.901 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0050096949999999998 2024-04-25T20:10:05.901 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.901 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.901 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:05.901 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132403+0000", 2024-04-25T20:10:05.901 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011332779999999999 2024-04-25T20:10:05.901 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.901 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.901 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:05.901 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132443+0000", 2024-04-25T20:10:05.901 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0175000000000001e-05 2024-04-25T20:10:05.901 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.902 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.902 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:05.902 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132471+0000", 2024-04-25T20:10:05.902 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8399e-05 2024-04-25T20:10:05.902 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.902 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:05.902 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.902 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.902 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.902 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:05.902 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:10:05.902 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 47.695488288999996, 2024-04-25T20:10:05.902 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011071002, 2024-04-25T20:10:05.902 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:05.903 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:05.903 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:05.903 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:05.903 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:05.903 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 141 2024-04-25T20:10:05.903 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.903 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:05.903 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.903 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:05.903 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:10:05.903 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.903 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.903 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.903 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:05.903 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:10:05.904 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.904 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.904 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.904 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:05.904 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121470+0000", 2024-04-25T20:10:05.904 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:05.904 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.904 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.904 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:05.904 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121473+0000", 2024-04-25T20:10:05.904 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.01e-06 2024-04-25T20:10:05.904 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.904 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.904 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:05.904 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121478+0000", 2024-04-25T20:10:05.905 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.515e-06 2024-04-25T20:10:05.905 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.905 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.905 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:05.905 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121480+0000", 2024-04-25T20:10:05.905 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5749999999999999e-06 2024-04-25T20:10:05.905 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.905 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.905 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:05.905 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126813+0000", 2024-04-25T20:10:05.905 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0053320399999999997 2024-04-25T20:10:05.905 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.905 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.905 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:05.905 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126854+0000", 2024-04-25T20:10:05.906 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1582999999999998e-05 2024-04-25T20:10:05.906 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.906 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.906 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:05.906 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126923+0000", 2024-04-25T20:10:05.906 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8544e-05 2024-04-25T20:10:05.906 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.906 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.906 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:05.906 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.131573+0000", 2024-04-25T20:10:05.906 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0046504659999999998 2024-04-25T20:10:05.906 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.906 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.906 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:05.907 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132478+0000", 2024-04-25T20:10:05.907 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00090442400000000003 2024-04-25T20:10:05.907 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.907 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.907 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:05.907 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132513+0000", 2024-04-25T20:10:05.907 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5204999999999999e-05 2024-04-25T20:10:05.907 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.907 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.907 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:05.907 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132542+0000", 2024-04-25T20:10:05.907 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.904e-05 2024-04-25T20:10:05.907 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.907 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:05.907 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.908 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.908 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.908 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:156 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:05.908 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:05.908 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 47.197663994000003, 2024-04-25T20:10:05.908 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090072280000000008, 2024-04-25T20:10:05.908 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:05.908 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:05.908 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:05.908 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:05.908 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:05.908 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 156 2024-04-25T20:10:05.908 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.908 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:05.908 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.908 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:05.908 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:05.909 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.909 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.909 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.909 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:05.909 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:05.909 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.909 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.909 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.909 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:05.909 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:05.909 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:05.909 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.909 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.909 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:05.909 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619297+0000", 2024-04-25T20:10:05.910 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2979999999999999e-06 2024-04-25T20:10:05.910 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.910 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.910 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:05.910 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619300+0000", 2024-04-25T20:10:05.910 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.371e-06 2024-04-25T20:10:05.910 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.910 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.910 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:05.910 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619304+0000", 2024-04-25T20:10:05.910 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.304e-06 2024-04-25T20:10:05.910 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.910 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.910 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:05.910 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.622910+0000", 2024-04-25T20:10:05.911 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0036065049999999999 2024-04-25T20:10:05.911 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.911 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.911 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:05.911 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.622990+0000", 2024-04-25T20:10:05.911 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.9722999999999995e-05 2024-04-25T20:10:05.911 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.911 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.911 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:05.911 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623128+0000", 2024-04-25T20:10:05.911 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00013818399999999999 2024-04-25T20:10:05.911 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.911 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.911 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:05.911 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625155+0000", 2024-04-25T20:10:05.912 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0020267979999999998 2024-04-25T20:10:05.912 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.912 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.912 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:05.912 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628225+0000", 2024-04-25T20:10:05.912 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0030705680000000001 2024-04-25T20:10:05.912 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.912 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.912 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:05.912 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628268+0000", 2024-04-25T20:10:05.912 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.2404000000000003e-05 2024-04-25T20:10:05.912 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.912 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.913 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:05.913 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628302+0000", 2024-04-25T20:10:05.913 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4439e-05 2024-04-25T20:10:05.913 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.913 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:05.913 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.913 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.913 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.913 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:157 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:05.913 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:05.913 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 47.197651895, 2024-04-25T20:10:05.913 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090534889999999996, 2024-04-25T20:10:05.913 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:05.913 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:05.913 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:05.914 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:05.914 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:05.914 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 157 2024-04-25T20:10:05.914 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.914 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:05.914 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.914 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:05.914 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:05.914 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.914 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.914 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.914 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:05.914 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:05.914 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.915 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.915 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.915 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:05.915 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:05.915 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:05.915 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.915 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.915 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:05.915 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619309+0000", 2024-04-25T20:10:05.915 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.841e-06 2024-04-25T20:10:05.915 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.915 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.915 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:05.915 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619312+0000", 2024-04-25T20:10:05.916 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.084e-06 2024-04-25T20:10:05.916 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.916 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.916 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:05.916 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619314+0000", 2024-04-25T20:10:05.916 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.3860000000000001e-06 2024-04-25T20:10:05.916 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.916 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.916 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:05.916 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623718+0000", 2024-04-25T20:10:05.916 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0044038050000000002 2024-04-25T20:10:05.916 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.916 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.916 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:05.917 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623791+0000", 2024-04-25T20:10:05.917 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3181000000000003e-05 2024-04-25T20:10:05.917 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.917 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.917 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:05.917 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623922+0000", 2024-04-25T20:10:05.917 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00013117000000000001 2024-04-25T20:10:05.917 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.917 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.917 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:05.917 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625187+0000", 2024-04-25T20:10:05.917 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001264961 2024-04-25T20:10:05.917 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.917 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.917 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:05.918 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628308+0000", 2024-04-25T20:10:05.918 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.003120739 2024-04-25T20:10:05.918 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.918 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.918 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:05.918 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628331+0000", 2024-04-25T20:10:05.918 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2802999999999998e-05 2024-04-25T20:10:05.918 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.918 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.918 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:05.918 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628361+0000", 2024-04-25T20:10:05.918 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9760999999999998e-05 2024-04-25T20:10:05.918 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.918 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:05.919 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.919 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.919 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.919 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:158 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:05.919 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:10:05.919 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 47.197641447999999, 2024-04-25T20:10:05.919 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0091076409999999997, 2024-04-25T20:10:05.919 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:05.919 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:05.919 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:05.919 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:05.919 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:05.919 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 158 2024-04-25T20:10:05.919 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.920 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:05.920 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.920 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:05.920 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:10:05.920 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.920 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.920 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.920 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:05.920 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:10:05.920 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.920 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.920 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.920 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:05.920 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619317+0000", 2024-04-25T20:10:05.920 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967296 2024-04-25T20:10:05.921 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.921 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.921 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:05.921 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619364+0000", 2024-04-25T20:10:05.921 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.6572000000000001e-05 2024-04-25T20:10:05.921 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.921 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.921 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:05.921 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619371+0000", 2024-04-25T20:10:05.921 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3860000000000001e-06 2024-04-25T20:10:05.921 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.921 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.921 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:05.922 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619382+0000", 2024-04-25T20:10:05.922 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0173999999999999e-05 2024-04-25T20:10:05.922 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.922 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.922 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:05.922 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624216+0000", 2024-04-25T20:10:05.922 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048340350000000004 2024-04-25T20:10:05.922 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.922 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.922 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:05.922 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624260+0000", 2024-04-25T20:10:05.922 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4558999999999998e-05 2024-04-25T20:10:05.922 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.922 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.923 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:05.923 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624333+0000", 2024-04-25T20:10:05.923 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2744999999999996e-05 2024-04-25T20:10:05.923 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.923 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.923 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:05.923 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625465+0000", 2024-04-25T20:10:05.923 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001132454 2024-04-25T20:10:05.923 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.923 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.923 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:05.923 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628365+0000", 2024-04-25T20:10:05.923 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0028994950000000002 2024-04-25T20:10:05.923 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.923 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.924 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:05.924 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628386+0000", 2024-04-25T20:10:05.924 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.1367000000000001e-05 2024-04-25T20:10:05.924 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.924 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.924 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:05.924 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628425+0000", 2024-04-25T20:10:05.924 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.9087e-05 2024-04-25T20:10:05.924 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.924 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:05.924 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.924 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.924 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.924 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:159 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:05.925 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:05.925 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 47.197572794999999, 2024-04-25T20:10:05.925 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090948999999999995, 2024-04-25T20:10:05.925 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:05.925 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:05.925 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:05.925 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:05.925 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:05.925 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 159 2024-04-25T20:10:05.925 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.925 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:05.925 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.925 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:05.925 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:05.926 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.926 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.926 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.926 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:05.926 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:05.926 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:05.926 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.926 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.926 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:05.926 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:05.926 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:05.926 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.926 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.926 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:05.926 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619389+0000", 2024-04-25T20:10:05.927 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.734e-06 2024-04-25T20:10:05.927 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.927 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.927 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:05.927 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619392+0000", 2024-04-25T20:10:05.927 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6219999999999998e-06 2024-04-25T20:10:05.927 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.927 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.927 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:05.927 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619403+0000", 2024-04-25T20:10:05.927 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0987e-05 2024-04-25T20:10:05.927 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.927 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.927 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:05.927 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624667+0000", 2024-04-25T20:10:05.928 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0052633970000000004 2024-04-25T20:10:05.928 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.928 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.928 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:05.928 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624741+0000", 2024-04-25T20:10:05.928 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.4583000000000001e-05 2024-04-25T20:10:05.928 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.928 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.928 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:05.928 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624857+0000", 2024-04-25T20:10:05.928 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000116118 2024-04-25T20:10:05.928 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.928 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.928 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:05.928 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.626013+0000", 2024-04-25T20:10:05.928 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001155473 2024-04-25T20:10:05.928 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.929 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.929 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:05.929 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628432+0000", 2024-04-25T20:10:05.929 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0024189379999999998 2024-04-25T20:10:05.929 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.929 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.929 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:05.929 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628454+0000", 2024-04-25T20:10:05.929 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2059999999999999e-05 2024-04-25T20:10:05.929 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:05.929 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:05.929 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:05.929 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628481+0000", 2024-04-25T20:10:05.929 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.7300999999999998e-05 2024-04-25T20:10:05.929 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.929 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:05.929 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.930 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:05.930 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:05.930 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:05.931 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:05.931+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:05.931 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T20:10:06.032 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:06.032+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:06.041 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:06.041 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:06.041 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:06.041 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:06.050 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T20:10:06.133 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:06.133+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:06.156 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:06.156 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:06.156 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:06.156 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:06.157 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:06.165 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T20:10:06.249 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:06.249+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:06.315 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:06.315 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:06.315 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:06.316 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:10:06.316 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:06.324 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T20:10:06.335 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:06.336+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:06.353 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is set 2024-04-25T20:10:06.437 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:06.437+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:06.469 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:06.470 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:06.470 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:06.470 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:06.479 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T20:10:06.538 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:06.538+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:06.586 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:06.586 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:06.586 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:06.586 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:06.586 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:06.595 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T20:10:06.639 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:06.639+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:06.706 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:06.706 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:06.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:06.706 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-04-25T20:10:06.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.706 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:30 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'16, mlcod=16'14)", 2024-04-25T20:10:06.707 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:10:06.707 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 49.537145316999997, 2024-04-25T20:10:06.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0054087670000000001, 2024-04-25T20:10:06.707 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:06.707 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:06.707 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:06.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:06.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:10:06.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.708 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:06.708 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:10:06.708 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.708 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:06.708 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167142+0000", 2024-04-25T20:10:06.708 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999976 2024-04-25T20:10:06.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.708 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:06.708 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167162+0000", 2024-04-25T20:10:06.709 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0109e-05 2024-04-25T20:10:06.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.709 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:06.709 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167188+0000", 2024-04-25T20:10:06.709 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6146e-05 2024-04-25T20:10:06.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.709 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:06.709 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167199+0000", 2024-04-25T20:10:06.709 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0767e-05 2024-04-25T20:10:06.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.709 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:06.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167256+0000", 2024-04-25T20:10:06.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7392000000000002e-05 2024-04-25T20:10:06.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:06.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167283+0000", 2024-04-25T20:10:06.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6577e-05 2024-04-25T20:10:06.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:06.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.172510+0000", 2024-04-25T20:10:06.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0052276759999999997 2024-04-25T20:10:06.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.711 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:06.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.172553+0000", 2024-04-25T20:10:06.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2444999999999998e-05 2024-04-25T20:10:06.711 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.711 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:06.711 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.711 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:31 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-04-25T20:10:06.711 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:06.711 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 49.536529239000004, 2024-04-25T20:10:06.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0055055479999999999, 2024-04-25T20:10:06.711 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:06.712 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:06.712 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:06.712 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.712 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:06.712 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:06.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.712 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.712 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:06.712 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:06.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.712 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.712 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:06.713 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:06.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:06.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.713 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:06.713 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167766+0000", 2024-04-25T20:10:06.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.122e-06 2024-04-25T20:10:06.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.713 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:06.713 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167776+0000", 2024-04-25T20:10:06.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0515e-05 2024-04-25T20:10:06.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:06.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167782+0000", 2024-04-25T20:10:06.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1149999999999999e-06 2024-04-25T20:10:06.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:06.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167837+0000", 2024-04-25T20:10:06.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4763999999999999e-05 2024-04-25T20:10:06.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:06.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167855+0000", 2024-04-25T20:10:06.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7589e-05 2024-04-25T20:10:06.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:06.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173221+0000", 2024-04-25T20:10:06.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.005365903 2024-04-25T20:10:06.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:06.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173266+0000", 2024-04-25T20:10:06.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5059999999999999e-05 2024-04-25T20:10:06.715 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.716 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:06.716 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.716 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:32 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-04-25T20:10:06.716 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:06.716 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 49.536154054000001, 2024-04-25T20:10:06.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0051558239999999998, 2024-04-25T20:10:06.716 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:06.716 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:06.716 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:06.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:06.717 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:06.717 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.717 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.717 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.717 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:06.717 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:06.717 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.717 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.717 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.717 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:06.718 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:06.718 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:06.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.718 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.718 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:06.718 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168173+0000", 2024-04-25T20:10:06.718 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7663999999999997e-05 2024-04-25T20:10:06.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.718 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.718 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:06.718 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168182+0000", 2024-04-25T20:10:06.718 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0650000000000005e-06 2024-04-25T20:10:06.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:06.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168188+0000", 2024-04-25T20:10:06.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0589999999999999e-06 2024-04-25T20:10:06.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:06.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168239+0000", 2024-04-25T20:10:06.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1467e-05 2024-04-25T20:10:06.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:06.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168260+0000", 2024-04-25T20:10:06.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0531e-05 2024-04-25T20:10:06.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:06.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173268+0000", 2024-04-25T20:10:06.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0050087980000000001 2024-04-25T20:10:06.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:06.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173291+0000", 2024-04-25T20:10:06.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.268e-05 2024-04-25T20:10:06.721 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.721 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:06.721 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.721 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.721 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:33 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'19, mlcod=16'14)", 2024-04-25T20:10:06.721 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:10:06.721 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 49.535643635, 2024-04-25T20:10:06.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046697049999999997, 2024-04-25T20:10:06.721 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:06.721 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:06.721 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:06.721 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.722 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:06.722 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:10:06.722 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.722 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.722 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.722 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:06.722 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:10:06.722 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.722 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.722 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.722 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:06.722 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168645+0000", 2024-04-25T20:10:06.722 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:06.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:06.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168651+0000", 2024-04-25T20:10:06.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1079999999999998e-06 2024-04-25T20:10:06.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:06.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168660+0000", 2024-04-25T20:10:06.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0010000000000007e-06 2024-04-25T20:10:06.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:06.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168666+0000", 2024-04-25T20:10:06.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7780000000000004e-06 2024-04-25T20:10:06.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:06.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168701+0000", 2024-04-25T20:10:06.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4490999999999999e-05 2024-04-25T20:10:06.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:06.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168729+0000", 2024-04-25T20:10:06.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8476999999999999e-05 2024-04-25T20:10:06.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:06.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173294+0000", 2024-04-25T20:10:06.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0045650730000000002 2024-04-25T20:10:06.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:06.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173315+0000", 2024-04-25T20:10:06.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1245000000000001e-05 2024-04-25T20:10:06.725 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.725 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:06.725 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.725 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:34 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'20, mlcod=16'14)", 2024-04-25T20:10:06.725 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:10:06.725 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 49.535144705999997, 2024-04-25T20:10:06.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041943309999999999, 2024-04-25T20:10:06.726 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:06.726 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:06.726 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:06.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.726 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:06.726 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:10:06.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.726 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:06.726 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:10:06.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.727 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:06.727 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169144+0000", 2024-04-25T20:10:06.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:06.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.727 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:06.727 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169150+0000", 2024-04-25T20:10:06.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9150000000000001e-06 2024-04-25T20:10:06.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.727 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:06.727 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169159+0000", 2024-04-25T20:10:06.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8899999999999996e-06 2024-04-25T20:10:06.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:06.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169169+0000", 2024-04-25T20:10:06.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0441e-05 2024-04-25T20:10:06.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:06.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169206+0000", 2024-04-25T20:10:06.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6816000000000001e-05 2024-04-25T20:10:06.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:06.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169234+0000", 2024-04-25T20:10:06.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8008999999999999e-05 2024-04-25T20:10:06.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:06.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173317+0000", 2024-04-25T20:10:06.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0040830160000000001 2024-04-25T20:10:06.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:06.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173339+0000", 2024-04-25T20:10:06.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1897000000000001e-05 2024-04-25T20:10:06.730 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.730 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:06.730 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.730 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.730 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:35 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'14)", 2024-04-25T20:10:06.731 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:10:06.731 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 49.534652387000001, 2024-04-25T20:10:06.731 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037236529999999999, 2024-04-25T20:10:06.731 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:06.731 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:06.731 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:06.731 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.731 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:06.732 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:10:06.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.732 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:06.732 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:10:06.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:06.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169636+0000", 2024-04-25T20:10:06.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:06.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:06.734 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169678+0000", 2024-04-25T20:10:06.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1752000000000003e-05 2024-04-25T20:10:06.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:06.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169688+0000", 2024-04-25T20:10:06.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0098000000000001e-05 2024-04-25T20:10:06.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:06.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169694+0000", 2024-04-25T20:10:06.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2349999999999996e-06 2024-04-25T20:10:06.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.736 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:06.736 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169733+0000", 2024-04-25T20:10:06.736 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8481e-05 2024-04-25T20:10:06.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.736 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:06.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169750+0000", 2024-04-25T20:10:06.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7509000000000001e-05 2024-04-25T20:10:06.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:06.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173340+0000", 2024-04-25T20:10:06.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0035898449999999999 2024-04-25T20:10:06.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.738 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:06.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173361+0000", 2024-04-25T20:10:06.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.039e-05 2024-04-25T20:10:06.739 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.739 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:06.739 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.739 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:36 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'22, mlcod=16'14)", 2024-04-25T20:10:06.740 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:06.740 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 49.534152327000001, 2024-04-25T20:10:06.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032472120000000002, 2024-04-25T20:10:06.740 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:06.740 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:06.740 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:06.740 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.740 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:06.741 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:06.741 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.741 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:06.741 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:06.741 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:06.742 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:06.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:06.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:06.742 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170152+0000", 2024-04-25T20:10:06.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5798e-05 2024-04-25T20:10:06.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:06.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170161+0000", 2024-04-25T20:10:06.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.064e-06 2024-04-25T20:10:06.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.743 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.743 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:06.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170167+0000", 2024-04-25T20:10:06.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8390000000000003e-06 2024-04-25T20:10:06.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.743 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.743 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:06.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170202+0000", 2024-04-25T20:10:06.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.5120999999999998e-05 2024-04-25T20:10:06.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.743 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.744 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:06.744 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170219+0000", 2024-04-25T20:10:06.744 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6929999999999999e-05 2024-04-25T20:10:06.744 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.744 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.744 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:06.744 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173362+0000", 2024-04-25T20:10:06.744 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0031427619999999999 2024-04-25T20:10:06.744 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.744 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.744 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:06.744 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173384+0000", 2024-04-25T20:10:06.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2237000000000001e-05 2024-04-25T20:10:06.745 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.745 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:06.745 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.745 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:40 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'26, mlcod=16'24)", 2024-04-25T20:10:06.745 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:10:06.745 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 49.527114415, 2024-04-25T20:10:06.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028727330000000001, 2024-04-25T20:10:06.745 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:06.745 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:06.745 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:06.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:06.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:10:06.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:06.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:10:06.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:06.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177173+0000", 2024-04-25T20:10:06.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999976 2024-04-25T20:10:06.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.747 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:06.747 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177186+0000", 2024-04-25T20:10:06.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3533000000000001e-05 2024-04-25T20:10:06.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.747 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:06.747 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177209+0000", 2024-04-25T20:10:06.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3280000000000001e-05 2024-04-25T20:10:06.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.748 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:06.748 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177220+0000", 2024-04-25T20:10:06.748 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0465000000000001e-05 2024-04-25T20:10:06.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.748 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.748 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:06.748 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177264+0000", 2024-04-25T20:10:06.748 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4413999999999999e-05 2024-04-25T20:10:06.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.748 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.748 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:06.749 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177289+0000", 2024-04-25T20:10:06.749 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4783999999999999e-05 2024-04-25T20:10:06.749 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.749 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.749 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:06.749 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.179990+0000", 2024-04-25T20:10:06.749 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027012939999999999 2024-04-25T20:10:06.749 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.749 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.749 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:06.749 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.180048+0000", 2024-04-25T20:10:06.749 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7299000000000001e-05 2024-04-25T20:10:06.749 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.750 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:06.750 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.750 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.750 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:67 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'41, mlcod=16'39)", 2024-04-25T20:10:06.750 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:10:06.750 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 49.492352922999999, 2024-04-25T20:10:06.750 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064192549999999996, 2024-04-25T20:10:06.750 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:06.750 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:06.750 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:06.750 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.750 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:06.751 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:10:06.751 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.751 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:06.751 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:10:06.751 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.751 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:06.751 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211935+0000", 2024-04-25T20:10:06.751 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999981 2024-04-25T20:10:06.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.752 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.752 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:06.752 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211948+0000", 2024-04-25T20:10:06.752 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3531e-05 2024-04-25T20:10:06.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.752 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.752 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:06.752 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211967+0000", 2024-04-25T20:10:06.752 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8326999999999999e-05 2024-04-25T20:10:06.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.752 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.752 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:06.753 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211975+0000", 2024-04-25T20:10:06.753 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8589999999999994e-06 2024-04-25T20:10:06.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.753 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.753 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:06.753 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212020+0000", 2024-04-25T20:10:06.753 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4626999999999998e-05 2024-04-25T20:10:06.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.753 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.753 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:06.753 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212045+0000", 2024-04-25T20:10:06.753 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4508999999999999e-05 2024-04-25T20:10:06.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.753 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.754 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:06.754 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218316+0000", 2024-04-25T20:10:06.754 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0062711640000000001 2024-04-25T20:10:06.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.754 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.754 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:06.754 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218356+0000", 2024-04-25T20:10:06.754 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9993000000000001e-05 2024-04-25T20:10:06.754 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.754 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:06.754 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.754 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.754 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:68 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-04-25T20:10:06.754 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:06.755 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 49.491903909000001, 2024-04-25T20:10:06.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011231365, 2024-04-25T20:10:06.755 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:06.755 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:06.755 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:06.755 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.755 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:06.755 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:06.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.755 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.755 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:06.755 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:06.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:06.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:06.756 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:06.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:06.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212392+0000", 2024-04-25T20:10:06.756 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.79e-06 2024-04-25T20:10:06.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:06.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212406+0000", 2024-04-25T20:10:06.757 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3577000000000001e-05 2024-04-25T20:10:06.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.757 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.757 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:06.757 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212414+0000", 2024-04-25T20:10:06.757 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6510000000000001e-06 2024-04-25T20:10:06.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.757 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.757 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:06.757 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217114+0000", 2024-04-25T20:10:06.757 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047005379999999998 2024-04-25T20:10:06.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.757 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.757 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:06.757 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217141+0000", 2024-04-25T20:10:06.758 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6984000000000001e-05 2024-04-25T20:10:06.758 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.758 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.758 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:06.758 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223563+0000", 2024-04-25T20:10:06.758 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064216999999999998 2024-04-25T20:10:06.758 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.758 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.758 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:06.758 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223617+0000", 2024-04-25T20:10:06.758 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4011000000000002e-05 2024-04-25T20:10:06.758 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.758 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:06.759 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.759 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.759 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.759 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:69 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-04-25T20:10:06.759 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:10:06.759 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 49.491371719, 2024-04-25T20:10:06.759 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.010730094000000001, 2024-04-25T20:10:06.759 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:06.759 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:06.759 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:06.759 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.759 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:06.759 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:10:06.760 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.760 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.760 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:06.760 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:10:06.760 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.760 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.760 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:06.760 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212917+0000", 2024-04-25T20:10:06.760 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:06.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.760 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.760 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:06.761 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212957+0000", 2024-04-25T20:10:06.761 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0692000000000003e-05 2024-04-25T20:10:06.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.761 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:06.761 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212973+0000", 2024-04-25T20:10:06.761 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.575e-05 2024-04-25T20:10:06.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.762 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:06.762 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212980+0000", 2024-04-25T20:10:06.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1679999999999998e-06 2024-04-25T20:10:06.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.762 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.762 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:06.762 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217449+0000", 2024-04-25T20:10:06.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.004468367 2024-04-25T20:10:06.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.762 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.763 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:06.763 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217470+0000", 2024-04-25T20:10:06.763 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0749000000000001e-05 2024-04-25T20:10:06.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.763 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.763 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:06.763 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223621+0000", 2024-04-25T20:10:06.763 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0061512880000000004 2024-04-25T20:10:06.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.763 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.763 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:06.763 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223648+0000", 2024-04-25T20:10:06.763 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6961000000000002e-05 2024-04-25T20:10:06.763 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.763 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:06.764 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.764 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.764 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:70 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'44, mlcod=16'39)", 2024-04-25T20:10:06.764 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:06.764 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 49.490612990999999, 2024-04-25T20:10:06.764 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.013553358, 2024-04-25T20:10:06.764 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:06.764 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:06.764 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:06.764 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.764 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:06.764 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:06.764 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.765 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.765 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:06.765 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:06.765 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.765 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.765 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.765 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:06.765 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:06.765 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:06.765 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.765 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.765 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:06.765 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213682+0000", 2024-04-25T20:10:06.765 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3439999999999997e-06 2024-04-25T20:10:06.765 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.766 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.766 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:06.766 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213693+0000", 2024-04-25T20:10:06.766 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1049999999999999e-05 2024-04-25T20:10:06.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.766 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.766 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:06.766 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213699+0000", 2024-04-25T20:10:06.766 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9399999999999999e-06 2024-04-25T20:10:06.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.766 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.766 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:06.766 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218086+0000", 2024-04-25T20:10:06.767 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0043871240000000001 2024-04-25T20:10:06.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.767 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.767 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:06.767 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218106+0000", 2024-04-25T20:10:06.767 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.997e-05 2024-04-25T20:10:06.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.767 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.767 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:06.767 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.227186+0000", 2024-04-25T20:10:06.767 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.009079574 2024-04-25T20:10:06.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.767 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.767 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:06.768 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.227230+0000", 2024-04-25T20:10:06.768 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4147999999999998e-05 2024-04-25T20:10:06.768 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.768 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:06.768 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.768 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.768 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:106 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-04-25T20:10:06.768 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:10:06.768 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 49.119473890999998, 2024-04-25T20:10:06.768 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002936756, 2024-04-25T20:10:06.768 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:06.768 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:06.769 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:06.769 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.769 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:06.769 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:10:06.769 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.769 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.769 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.769 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:06.769 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:10:06.769 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.769 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.769 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.769 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:06.769 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584815+0000", 2024-04-25T20:10:06.770 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:06.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.770 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.770 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:06.770 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584860+0000", 2024-04-25T20:10:06.770 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5161999999999999e-05 2024-04-25T20:10:06.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.770 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.770 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:06.770 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584874+0000", 2024-04-25T20:10:06.770 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4464000000000001e-05 2024-04-25T20:10:06.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.771 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.771 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:06.771 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584881+0000", 2024-04-25T20:10:06.771 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.409e-06 2024-04-25T20:10:06.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.771 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.771 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:06.771 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584915+0000", 2024-04-25T20:10:06.771 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4007000000000002e-05 2024-04-25T20:10:06.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.771 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.771 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:06.771 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584933+0000", 2024-04-25T20:10:06.772 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7737999999999999e-05 2024-04-25T20:10:06.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.772 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.772 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:06.772 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587717+0000", 2024-04-25T20:10:06.772 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002784711 2024-04-25T20:10:06.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.772 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.772 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:06.772 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587752+0000", 2024-04-25T20:10:06.772 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4915e-05 2024-04-25T20:10:06.773 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.773 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:06.773 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.773 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.773 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:107 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'69, mlcod=16'64)", 2024-04-25T20:10:06.773 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:10:06.773 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 49.119398588999999, 2024-04-25T20:10:06.773 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028906270000000002, 2024-04-25T20:10:06.773 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:06.773 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:06.773 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:06.773 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.773 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:06.774 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:10:06.774 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.774 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.774 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:06.774 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:10:06.774 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.774 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.774 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:06.774 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584890+0000", 2024-04-25T20:10:06.774 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:06.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.775 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:06.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584895+0000", 2024-04-25T20:10:06.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2069999999999997e-06 2024-04-25T20:10:06.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.775 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:06.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584906+0000", 2024-04-25T20:10:06.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.117e-05 2024-04-25T20:10:06.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:06.776 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584910+0000", 2024-04-25T20:10:06.776 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4190000000000004e-06 2024-04-25T20:10:06.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:06.776 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585310+0000", 2024-04-25T20:10:06.776 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00040002700000000001 2024-04-25T20:10:06.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:06.776 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585329+0000", 2024-04-25T20:10:06.776 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8838000000000002e-05 2024-04-25T20:10:06.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.777 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:06.777 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587754+0000", 2024-04-25T20:10:06.777 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024252689999999999 2024-04-25T20:10:06.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.777 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.777 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:06.777 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587781+0000", 2024-04-25T20:10:06.777 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7206999999999999e-05 2024-04-25T20:10:06.777 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.777 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:06.777 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.778 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.778 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:108 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'70, mlcod=16'64)", 2024-04-25T20:10:06.778 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:10:06.778 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 49.119373437999997, 2024-04-25T20:10:06.778 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028924879999999999, 2024-04-25T20:10:06.778 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:06.778 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:06.778 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:06.778 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.778 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:06.778 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:10:06.778 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.779 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.779 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:06.779 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:10:06.779 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.779 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.779 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:06.779 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:10:06.779 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:06.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.779 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.779 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:06.779 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584919+0000", 2024-04-25T20:10:06.779 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.557e-06 2024-04-25T20:10:06.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.780 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.780 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:06.780 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584925+0000", 2024-04-25T20:10:06.780 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8619999999999999e-06 2024-04-25T20:10:06.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.780 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.780 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:06.780 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584931+0000", 2024-04-25T20:10:06.780 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2539999999999998e-06 2024-04-25T20:10:06.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.780 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.780 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:06.780 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585550+0000", 2024-04-25T20:10:06.781 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00061821800000000002 2024-04-25T20:10:06.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.781 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.781 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:06.781 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585567+0000", 2024-04-25T20:10:06.781 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7499e-05 2024-04-25T20:10:06.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.781 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.781 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:06.781 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587783+0000", 2024-04-25T20:10:06.781 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0022156200000000002 2024-04-25T20:10:06.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.781 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.781 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:06.781 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587808+0000", 2024-04-25T20:10:06.782 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5729e-05 2024-04-25T20:10:06.782 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.782 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:06.782 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.782 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.782 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.782 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:112 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'74, mlcod=16'66)", 2024-04-25T20:10:06.782 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:10:06.782 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 49.117215815999998, 2024-04-25T20:10:06.782 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00318724, 2024-04-25T20:10:06.782 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:06.782 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:06.782 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:06.783 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.783 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:06.783 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:10:06.783 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.783 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.783 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.783 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:06.783 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:10:06.783 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.783 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.783 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.783 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:06.784 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587073+0000", 2024-04-25T20:10:06.784 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:06.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.784 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.784 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:06.784 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587136+0000", 2024-04-25T20:10:06.784 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2928999999999993e-05 2024-04-25T20:10:06.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.784 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.784 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:06.784 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587147+0000", 2024-04-25T20:10:06.784 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1182999999999999e-05 2024-04-25T20:10:06.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.785 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:06.785 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587153+0000", 2024-04-25T20:10:06.785 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9059999999999999e-06 2024-04-25T20:10:06.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.785 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:06.785 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587306+0000", 2024-04-25T20:10:06.785 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000152848 2024-04-25T20:10:06.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.785 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:06.785 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587330+0000", 2024-04-25T20:10:06.785 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4304e-05 2024-04-25T20:10:06.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.786 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.786 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:06.786 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.590230+0000", 2024-04-25T20:10:06.786 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029004629999999998 2024-04-25T20:10:06.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.786 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.786 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:06.786 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.590261+0000", 2024-04-25T20:10:06.786 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0323000000000001e-05 2024-04-25T20:10:06.786 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.786 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:06.786 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.787 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.787 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:118 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'80, mlcod=16'78)", 2024-04-25T20:10:06.787 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:10:06.787 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 49.112429237000001, 2024-04-25T20:10:06.787 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029618959999999999, 2024-04-25T20:10:06.787 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:06.787 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:06.787 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:06.787 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.787 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:06.787 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:10:06.787 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.787 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.787 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:06.788 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:10:06.788 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.788 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.788 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.788 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:06.788 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591859+0000", 2024-04-25T20:10:06.788 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:06.788 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.788 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.788 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:06.788 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591918+0000", 2024-04-25T20:10:06.788 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8842999999999999e-05 2024-04-25T20:10:06.788 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.788 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.789 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:06.789 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591933+0000", 2024-04-25T20:10:06.789 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4600000000000001e-05 2024-04-25T20:10:06.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.789 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.789 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:06.789 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591939+0000", 2024-04-25T20:10:06.789 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4559999999999997e-06 2024-04-25T20:10:06.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.789 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.789 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:06.789 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591990+0000", 2024-04-25T20:10:06.789 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0318999999999997e-05 2024-04-25T20:10:06.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.789 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.790 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:06.790 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.592008+0000", 2024-04-25T20:10:06.790 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8819000000000001e-05 2024-04-25T20:10:06.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.790 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.790 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:06.790 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.594784+0000", 2024-04-25T20:10:06.790 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027750850000000001 2024-04-25T20:10:06.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.790 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.790 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:06.790 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.594822+0000", 2024-04-25T20:10:06.790 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8522000000000001e-05 2024-04-25T20:10:06.790 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.791 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:06.791 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.791 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.791 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.791 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:139 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'97, mlcod=17'87)", 2024-04-25T20:10:06.791 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:10:06.791 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 48.578392632000003, 2024-04-25T20:10:06.791 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048546290000000001, 2024-04-25T20:10:06.791 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:06.791 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:06.791 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:06.791 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.791 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:06.791 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:10:06.792 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.792 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.792 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:06.792 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:10:06.792 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.792 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.792 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:06.792 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125896+0000", 2024-04-25T20:10:06.792 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:06.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.792 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.792 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:06.792 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125908+0000", 2024-04-25T20:10:06.793 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.137e-05 2024-04-25T20:10:06.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.793 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.793 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:06.793 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125915+0000", 2024-04-25T20:10:06.793 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3939999999999998e-06 2024-04-25T20:10:06.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.793 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.793 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:06.793 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125922+0000", 2024-04-25T20:10:06.793 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1930000000000004e-06 2024-04-25T20:10:06.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.793 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.793 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:06.793 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125959+0000", 2024-04-25T20:10:06.794 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6701e-05 2024-04-25T20:10:06.794 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.794 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.794 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:06.794 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125972+0000", 2024-04-25T20:10:06.794 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2578000000000001e-05 2024-04-25T20:10:06.794 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.794 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.794 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:06.794 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130719+0000", 2024-04-25T20:10:06.794 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047475800000000004 2024-04-25T20:10:06.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.795 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.795 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:06.795 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130751+0000", 2024-04-25T20:10:06.795 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2283000000000002e-05 2024-04-25T20:10:06.795 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.795 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:06.795 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.795 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.795 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:140 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'98, mlcod=17'87)", 2024-04-25T20:10:06.796 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:06.796 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 48.577967280999999, 2024-04-25T20:10:06.796 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048310990000000002, 2024-04-25T20:10:06.796 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:06.796 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:06.796 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:06.796 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.796 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:06.796 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:06.796 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.796 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.796 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:06.796 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:06.796 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.797 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:06.797 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:06.797 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:06.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.797 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:06.797 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126427+0000", 2024-04-25T20:10:06.797 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000105572 2024-04-25T20:10:06.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.797 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:06.797 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126435+0000", 2024-04-25T20:10:06.798 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1920000000000005e-06 2024-04-25T20:10:06.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.798 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:06.798 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126439+0000", 2024-04-25T20:10:06.798 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6950000000000002e-06 2024-04-25T20:10:06.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.798 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:06.798 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126471+0000", 2024-04-25T20:10:06.798 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1699000000000003e-05 2024-04-25T20:10:06.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.799 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:06.799 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126484+0000", 2024-04-25T20:10:06.799 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.343e-05 2024-04-25T20:10:06.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.799 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.799 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:06.799 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131131+0000", 2024-04-25T20:10:06.799 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046471480000000003 2024-04-25T20:10:06.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.799 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.799 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:06.799 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131153+0000", 2024-04-25T20:10:06.800 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1753e-05 2024-04-25T20:10:06.800 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.800 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:06.800 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.800 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.800 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:141 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'99, mlcod=17'87)", 2024-04-25T20:10:06.800 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:06.800 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 48.577312337000002, 2024-04-25T20:10:06.800 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.004496847, 2024-04-25T20:10:06.800 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:06.800 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:06.800 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:06.800 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.800 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:06.801 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:06.801 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.801 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.801 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:06.801 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:06.801 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:06.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.801 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.801 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:06.801 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:06.801 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:06.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.801 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.801 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:06.802 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126980+0000", 2024-04-25T20:10:06.802 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.619e-06 2024-04-25T20:10:06.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.802 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.802 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:06.802 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126987+0000", 2024-04-25T20:10:06.802 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7220000000000002e-06 2024-04-25T20:10:06.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.802 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.802 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:06.802 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126992+0000", 2024-04-25T20:10:06.802 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8679999999999998e-06 2024-04-25T20:10:06.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.802 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.803 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:06.803 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130772+0000", 2024-04-25T20:10:06.803 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037803120000000001 2024-04-25T20:10:06.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.803 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.803 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:06.803 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130786+0000", 2024-04-25T20:10:06.803 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3592e-05 2024-04-25T20:10:06.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.803 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.803 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:06.803 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131451+0000", 2024-04-25T20:10:06.803 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000665206 2024-04-25T20:10:06.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:06.803 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:06.804 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:06.804 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131474+0000", 2024-04-25T20:10:06.804 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2957999999999999e-05 2024-04-25T20:10:06.804 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.804 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:06.804 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.804 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:06.804 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:06.804 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:06.806 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T20:10:06.806 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:06.806+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:06.907 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:06.907+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:06.955 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:06.955 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:06.956 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:06.956 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:06.965 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T20:10:07.008 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:07.008+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:07.072 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:07.072 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:07.072 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:07.072 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:07.072 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:07.081 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T20:10:07.124 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:07.124+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:07.190 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:07.190 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:07.190 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:07.190 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:10:07.190 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:07.199 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T20:10:07.241 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:07.241+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:07.311 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:07.311+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:07.335 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T20:10:07.349 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6, 7] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-25T20:10:07.350 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T20:10:07.350 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [0, 1, 2, 4, 5, 6, 7] 2024-04-25T20:10:07.350 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-04-25T20:10:07.352 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:07.352 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:07.352 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:07.352 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:07.364 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T20:10:07.450 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:07.450+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:07.477 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:10:07.477 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:10:07.496 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:10:07.497 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:10:07.513 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:07.513+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:07.514 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:10:07.514 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:10:07.527 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:07.527 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:07.527 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:07.528 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:07.528 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:07.537 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T20:10:07.539 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:10:07.539 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:10:07.559 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:10:07.559 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:10:07.577 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:10:07.577 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:10:07.595 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:10:07.596 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:10:07.614 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:07.614+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:07.615 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:10:07.615 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:10:07.656 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:07.656 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:07.666 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T20:10:07.693 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-04-25T20:10:07.693+0000 7f249d8be640 -1 mon.a@0(leader).osd e36 definitely_dead 0 2024-04-25T20:10:07.715 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:07.715+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:07.779 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:07.779 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:07.788 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T20:10:07.816 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:07.815+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:07.917 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:07.917+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:07.931 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:07.932 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:07.940 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T20:10:08.018 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:08.017+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:08.081 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:08.081 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:08.090 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T20:10:08.135 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:08.135+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:08.191 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:08.191 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:08.200 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T20:10:08.241 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:08.241+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:08.321 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:08.321+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:08.341 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:08.341 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:08.349 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T20:10:08.367 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-25T20:10:08.372 INFO:teuthology.orchestra.run.smithi039.stderr:marked out osd.2. 2024-04-25T20:10:08.429 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:08.429+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:08.454 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:08.454 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:08.459 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T20:10:08.522 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:08.523+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:08.562 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:08.562 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:08.570 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T20:10:08.657 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:08.657+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:08.714 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:08.715 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:08.723 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T20:10:08.724 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:08.724+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:08.825 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:08.825+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:08.866 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:08.866 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:08.873 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T20:10:08.927 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:08.927+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:08.974 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:08.974 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:08.983 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T20:10:09.027 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:09.027+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:09.124 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:09.125 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:09.128 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:09.128+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:09.133 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T20:10:09.228 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:09.229+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:09.275 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:09.275 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:09.284 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T20:10:09.330 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:09.329+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:09.388 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is unset 2024-04-25T20:10:09.407 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:09.407 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:09.417 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T20:10:09.458 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:09.458+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:09.531 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:09.531+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:09.567 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:09.567 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:09.576 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T20:10:09.630 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T20:10:09.631 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:09.632+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:09.690 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:09.690 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:09.699 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T20:10:09.732 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:09.733+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:09.769 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:10:09.769 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:10:09.786 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:10:09.787 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:10:09.819 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:10:09.819 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:10:09.829 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:09.829 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:09.830 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:09.830 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:09.833 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:09.832+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:09.839 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T20:10:09.840 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:10:09.840 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:10:09.857 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:10:09.857 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:10:09.875 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:10:09.875 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:10:09.894 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:10:09.894 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:10:09.909 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:10:09.910 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:10:09.963 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:09.963+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:09.969 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:09.969 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:09.969 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:09.969 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:09.969 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:09.978 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T20:10:10.035 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:10.035+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:10.128 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:10.128 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:10.128 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:10.128 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-04-25T20:10:10.128 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.128 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:30 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'16, mlcod=16'14)", 2024-04-25T20:10:10.128 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:10:10.128 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 52.959885223999997, 2024-04-25T20:10:10.129 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0054087670000000001, 2024-04-25T20:10:10.129 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:10.129 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:10.129 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:10.129 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.129 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:10.129 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:10:10.129 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.129 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.129 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:10.129 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:10:10.130 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.130 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.130 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.130 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:10.130 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167142+0000", 2024-04-25T20:10:10.130 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999976 2024-04-25T20:10:10.130 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.130 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.130 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:10.130 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167162+0000", 2024-04-25T20:10:10.130 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0109e-05 2024-04-25T20:10:10.130 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.130 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.130 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:10.130 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167188+0000", 2024-04-25T20:10:10.131 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6146e-05 2024-04-25T20:10:10.131 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.131 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.131 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:10.131 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167199+0000", 2024-04-25T20:10:10.131 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0767e-05 2024-04-25T20:10:10.131 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.131 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.131 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:10.131 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167256+0000", 2024-04-25T20:10:10.131 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7392000000000002e-05 2024-04-25T20:10:10.131 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.132 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.132 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:10.132 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167283+0000", 2024-04-25T20:10:10.132 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6577e-05 2024-04-25T20:10:10.132 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.132 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.132 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:10.132 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.172510+0000", 2024-04-25T20:10:10.132 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0052276759999999997 2024-04-25T20:10:10.132 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.132 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.132 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:10.132 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.172553+0000", 2024-04-25T20:10:10.132 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2444999999999998e-05 2024-04-25T20:10:10.132 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.133 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:10.133 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.133 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.133 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:31 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-04-25T20:10:10.133 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:10.133 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 52.959269145999997, 2024-04-25T20:10:10.133 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0055055479999999999, 2024-04-25T20:10:10.133 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:10.133 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:10.133 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:10.133 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.133 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:10.134 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:10.134 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.134 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.134 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:10.134 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:10.134 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.134 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.134 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:10.134 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:10.134 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:10.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.134 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.135 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:10.135 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167766+0000", 2024-04-25T20:10:10.135 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.122e-06 2024-04-25T20:10:10.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.135 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.135 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:10.135 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167776+0000", 2024-04-25T20:10:10.135 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0515e-05 2024-04-25T20:10:10.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.135 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.135 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:10.135 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167782+0000", 2024-04-25T20:10:10.135 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1149999999999999e-06 2024-04-25T20:10:10.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.136 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.136 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:10.136 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167837+0000", 2024-04-25T20:10:10.136 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4763999999999999e-05 2024-04-25T20:10:10.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.136 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.136 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:10.136 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167855+0000", 2024-04-25T20:10:10.136 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7589e-05 2024-04-25T20:10:10.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.136 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.136 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:10.136 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173221+0000", 2024-04-25T20:10:10.136 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.005365903 2024-04-25T20:10:10.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.137 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.137 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:10.137 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173266+0000", 2024-04-25T20:10:10.137 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5059999999999999e-05 2024-04-25T20:10:10.137 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.137 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:10.137 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.137 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.137 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.137 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:32 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-04-25T20:10:10.137 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:10.137 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 52.958893961000001, 2024-04-25T20:10:10.137 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0051558239999999998, 2024-04-25T20:10:10.137 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:10.137 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:10.138 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:10.138 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.138 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:10.138 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:10.138 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.138 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.138 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:10.138 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:10.138 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.138 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.138 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:10.138 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:10.139 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:10.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.139 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.139 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:10.139 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168173+0000", 2024-04-25T20:10:10.139 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7663999999999997e-05 2024-04-25T20:10:10.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.139 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.139 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:10.139 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168182+0000", 2024-04-25T20:10:10.139 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0650000000000005e-06 2024-04-25T20:10:10.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.139 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.140 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:10.140 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168188+0000", 2024-04-25T20:10:10.140 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0589999999999999e-06 2024-04-25T20:10:10.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.140 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.140 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:10.140 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168239+0000", 2024-04-25T20:10:10.140 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1467e-05 2024-04-25T20:10:10.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.140 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.140 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:10.140 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168260+0000", 2024-04-25T20:10:10.140 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0531e-05 2024-04-25T20:10:10.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.141 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.141 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:10.141 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173268+0000", 2024-04-25T20:10:10.141 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0050087980000000001 2024-04-25T20:10:10.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.141 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.141 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:10.141 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173291+0000", 2024-04-25T20:10:10.141 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.268e-05 2024-04-25T20:10:10.141 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.141 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:10.141 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.141 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.141 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:33 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'19, mlcod=16'14)", 2024-04-25T20:10:10.142 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:10:10.142 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 52.958383542, 2024-04-25T20:10:10.142 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046697049999999997, 2024-04-25T20:10:10.142 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:10.142 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:10.142 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:10.142 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.142 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:10.142 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:10:10.142 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.142 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.142 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.142 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:10.142 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:10:10.142 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168645+0000", 2024-04-25T20:10:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168651+0000", 2024-04-25T20:10:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1079999999999998e-06 2024-04-25T20:10:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168660+0000", 2024-04-25T20:10:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0010000000000007e-06 2024-04-25T20:10:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.144 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.144 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:10.144 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168666+0000", 2024-04-25T20:10:10.144 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7780000000000004e-06 2024-04-25T20:10:10.144 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.144 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.144 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:10.144 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168701+0000", 2024-04-25T20:10:10.144 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4490999999999999e-05 2024-04-25T20:10:10.144 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.144 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.144 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:10.144 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168729+0000", 2024-04-25T20:10:10.144 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8476999999999999e-05 2024-04-25T20:10:10.144 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.145 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.145 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:10.145 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173294+0000", 2024-04-25T20:10:10.145 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0045650730000000002 2024-04-25T20:10:10.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.145 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.145 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:10.145 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173315+0000", 2024-04-25T20:10:10.145 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1245000000000001e-05 2024-04-25T20:10:10.145 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.145 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:10.145 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.146 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.146 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:34 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'20, mlcod=16'14)", 2024-04-25T20:10:10.146 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:10:10.146 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 52.957884612999997, 2024-04-25T20:10:10.146 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041943309999999999, 2024-04-25T20:10:10.146 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:10.146 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:10.146 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:10.146 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.146 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:10.146 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:10:10.146 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.146 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:10:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169144+0000", 2024-04-25T20:10:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169150+0000", 2024-04-25T20:10:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9150000000000001e-06 2024-04-25T20:10:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169159+0000", 2024-04-25T20:10:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8899999999999996e-06 2024-04-25T20:10:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169169+0000", 2024-04-25T20:10:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0441e-05 2024-04-25T20:10:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169206+0000", 2024-04-25T20:10:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6816000000000001e-05 2024-04-25T20:10:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:10.149 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169234+0000", 2024-04-25T20:10:10.149 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8008999999999999e-05 2024-04-25T20:10:10.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.149 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.149 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:10.149 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173317+0000", 2024-04-25T20:10:10.149 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0040830160000000001 2024-04-25T20:10:10.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.149 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.149 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:10.149 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173339+0000", 2024-04-25T20:10:10.149 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1897000000000001e-05 2024-04-25T20:10:10.149 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.150 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:10.150 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.150 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.150 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.150 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:35 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'14)", 2024-04-25T20:10:10.150 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:10:10.150 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 52.957392294000002, 2024-04-25T20:10:10.150 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037236529999999999, 2024-04-25T20:10:10.150 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:10.150 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:10.150 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:10.150 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.150 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:10.150 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:10:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:10:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169636+0000", 2024-04-25T20:10:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169678+0000", 2024-04-25T20:10:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1752000000000003e-05 2024-04-25T20:10:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169688+0000", 2024-04-25T20:10:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0098000000000001e-05 2024-04-25T20:10:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169694+0000", 2024-04-25T20:10:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2349999999999996e-06 2024-04-25T20:10:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169733+0000", 2024-04-25T20:10:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8481e-05 2024-04-25T20:10:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.153 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:10.153 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169750+0000", 2024-04-25T20:10:10.153 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7509000000000001e-05 2024-04-25T20:10:10.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.153 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.153 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:10.153 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173340+0000", 2024-04-25T20:10:10.153 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0035898449999999999 2024-04-25T20:10:10.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.153 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.153 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:10.153 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173361+0000", 2024-04-25T20:10:10.153 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.039e-05 2024-04-25T20:10:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:36 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'22, mlcod=16'14)", 2024-04-25T20:10:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 52.956892234000001, 2024-04-25T20:10:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032472120000000002, 2024-04-25T20:10:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170152+0000", 2024-04-25T20:10:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5798e-05 2024-04-25T20:10:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170161+0000", 2024-04-25T20:10:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.064e-06 2024-04-25T20:10:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170167+0000", 2024-04-25T20:10:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8390000000000003e-06 2024-04-25T20:10:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170202+0000", 2024-04-25T20:10:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.5120999999999998e-05 2024-04-25T20:10:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:10.157 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170219+0000", 2024-04-25T20:10:10.157 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6929999999999999e-05 2024-04-25T20:10:10.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.157 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.157 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:10.157 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173362+0000", 2024-04-25T20:10:10.157 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0031427619999999999 2024-04-25T20:10:10.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.157 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.157 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:10.157 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173384+0000", 2024-04-25T20:10:10.157 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2237000000000001e-05 2024-04-25T20:10:10.157 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.157 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:10.158 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.158 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.158 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:40 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'26, mlcod=16'24)", 2024-04-25T20:10:10.158 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:10:10.158 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 52.949854322, 2024-04-25T20:10:10.158 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028727330000000001, 2024-04-25T20:10:10.158 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:10.158 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:10.158 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:10.158 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.158 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:10.158 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:10:10.158 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.159 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.159 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:10.159 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:10:10.159 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.159 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.159 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:10.159 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177173+0000", 2024-04-25T20:10:10.159 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999976 2024-04-25T20:10:10.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.159 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.159 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:10.159 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177186+0000", 2024-04-25T20:10:10.160 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3533000000000001e-05 2024-04-25T20:10:10.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.160 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.160 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:10.160 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177209+0000", 2024-04-25T20:10:10.160 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3280000000000001e-05 2024-04-25T20:10:10.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.160 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.160 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:10.160 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177220+0000", 2024-04-25T20:10:10.160 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0465000000000001e-05 2024-04-25T20:10:10.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.160 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.160 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:10.161 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177264+0000", 2024-04-25T20:10:10.161 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4413999999999999e-05 2024-04-25T20:10:10.161 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.161 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.161 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:10.161 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177289+0000", 2024-04-25T20:10:10.161 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4783999999999999e-05 2024-04-25T20:10:10.161 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.161 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.161 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:10.161 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.179990+0000", 2024-04-25T20:10:10.161 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027012939999999999 2024-04-25T20:10:10.161 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.161 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.162 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:10.162 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.180048+0000", 2024-04-25T20:10:10.162 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7299000000000001e-05 2024-04-25T20:10:10.162 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.162 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:10.162 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.162 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.162 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:67 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'41, mlcod=16'39)", 2024-04-25T20:10:10.162 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:10:10.162 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 52.915092829999999, 2024-04-25T20:10:10.162 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064192549999999996, 2024-04-25T20:10:10.162 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:10.162 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:10.162 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:10.163 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.163 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:10.163 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:10:10.163 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.163 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.163 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:10.163 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:10:10.163 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.163 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.163 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:10.163 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211935+0000", 2024-04-25T20:10:10.163 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999981 2024-04-25T20:10:10.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.164 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.164 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:10.164 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211948+0000", 2024-04-25T20:10:10.164 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3531e-05 2024-04-25T20:10:10.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.164 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.164 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:10.164 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211967+0000", 2024-04-25T20:10:10.164 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8326999999999999e-05 2024-04-25T20:10:10.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.164 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.164 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:10.164 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211975+0000", 2024-04-25T20:10:10.164 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8589999999999994e-06 2024-04-25T20:10:10.165 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.165 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.165 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:10.165 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212020+0000", 2024-04-25T20:10:10.165 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4626999999999998e-05 2024-04-25T20:10:10.165 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.165 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.165 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:10.165 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212045+0000", 2024-04-25T20:10:10.165 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4508999999999999e-05 2024-04-25T20:10:10.165 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.165 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.165 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:10.166 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218316+0000", 2024-04-25T20:10:10.166 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0062711640000000001 2024-04-25T20:10:10.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.166 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.166 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:10.166 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218356+0000", 2024-04-25T20:10:10.166 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9993000000000001e-05 2024-04-25T20:10:10.166 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.166 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:10.166 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.166 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.166 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:68 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-04-25T20:10:10.167 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:10.167 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 52.914643816000002, 2024-04-25T20:10:10.167 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011231365, 2024-04-25T20:10:10.167 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:10.167 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:10.167 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:10.167 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.167 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:10.167 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:10.167 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.167 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.167 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:10.167 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:10.168 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.168 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.168 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:10.168 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:10.168 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:10.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.168 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.168 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:10.168 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212392+0000", 2024-04-25T20:10:10.168 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.79e-06 2024-04-25T20:10:10.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.168 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.168 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:10.169 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212406+0000", 2024-04-25T20:10:10.169 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3577000000000001e-05 2024-04-25T20:10:10.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.169 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.169 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:10.169 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212414+0000", 2024-04-25T20:10:10.169 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6510000000000001e-06 2024-04-25T20:10:10.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.169 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.169 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:10.169 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217114+0000", 2024-04-25T20:10:10.169 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047005379999999998 2024-04-25T20:10:10.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.170 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.170 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:10.170 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217141+0000", 2024-04-25T20:10:10.170 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6984000000000001e-05 2024-04-25T20:10:10.170 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.170 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.170 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:10.170 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223563+0000", 2024-04-25T20:10:10.170 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064216999999999998 2024-04-25T20:10:10.170 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.170 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.170 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:10.170 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223617+0000", 2024-04-25T20:10:10.170 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4011000000000002e-05 2024-04-25T20:10:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:69 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-04-25T20:10:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:10:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 52.914111626, 2024-04-25T20:10:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.010730094000000001, 2024-04-25T20:10:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:10:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:10:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212917+0000", 2024-04-25T20:10:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212957+0000", 2024-04-25T20:10:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0692000000000003e-05 2024-04-25T20:10:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.173 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.173 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:10.173 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212973+0000", 2024-04-25T20:10:10.173 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.575e-05 2024-04-25T20:10:10.173 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.173 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.173 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:10.173 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212980+0000", 2024-04-25T20:10:10.173 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1679999999999998e-06 2024-04-25T20:10:10.173 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.173 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.173 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:10.173 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217449+0000", 2024-04-25T20:10:10.173 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.004468367 2024-04-25T20:10:10.173 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.173 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.174 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:10.174 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217470+0000", 2024-04-25T20:10:10.174 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0749000000000001e-05 2024-04-25T20:10:10.174 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.174 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.174 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:10.174 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223621+0000", 2024-04-25T20:10:10.174 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0061512880000000004 2024-04-25T20:10:10.174 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.174 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.174 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:10.174 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223648+0000", 2024-04-25T20:10:10.174 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6961000000000002e-05 2024-04-25T20:10:10.174 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:70 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'44, mlcod=16'39)", 2024-04-25T20:10:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 52.913352897999999, 2024-04-25T20:10:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.013553358, 2024-04-25T20:10:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.176 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:10.176 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:10.176 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.176 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.176 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.176 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:10.176 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:10.176 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:10.176 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.176 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.176 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:10.176 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213682+0000", 2024-04-25T20:10:10.176 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3439999999999997e-06 2024-04-25T20:10:10.176 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.176 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.176 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213693+0000", 2024-04-25T20:10:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1049999999999999e-05 2024-04-25T20:10:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213699+0000", 2024-04-25T20:10:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9399999999999999e-06 2024-04-25T20:10:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218086+0000", 2024-04-25T20:10:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0043871240000000001 2024-04-25T20:10:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218106+0000", 2024-04-25T20:10:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.997e-05 2024-04-25T20:10:10.178 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.178 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.178 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:10.178 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.227186+0000", 2024-04-25T20:10:10.178 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.009079574 2024-04-25T20:10:10.178 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.178 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.178 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:10.178 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.227230+0000", 2024-04-25T20:10:10.178 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4147999999999998e-05 2024-04-25T20:10:10.178 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.178 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:10.178 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.178 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.178 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.179 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:106 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-04-25T20:10:10.179 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:10:10.179 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 52.542213797999999, 2024-04-25T20:10:10.179 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002936756, 2024-04-25T20:10:10.179 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:10.179 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:10.179 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:10.179 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.179 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:10.179 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:10:10.179 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.179 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.179 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.179 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:10.179 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:10:10.179 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584815+0000", 2024-04-25T20:10:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584860+0000", 2024-04-25T20:10:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5161999999999999e-05 2024-04-25T20:10:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584874+0000", 2024-04-25T20:10:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4464000000000001e-05 2024-04-25T20:10:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584881+0000", 2024-04-25T20:10:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.409e-06 2024-04-25T20:10:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584915+0000", 2024-04-25T20:10:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4007000000000002e-05 2024-04-25T20:10:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584933+0000", 2024-04-25T20:10:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7737999999999999e-05 2024-04-25T20:10:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:10.182 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587717+0000", 2024-04-25T20:10:10.182 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002784711 2024-04-25T20:10:10.182 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.182 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.182 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:10.182 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587752+0000", 2024-04-25T20:10:10.182 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4915e-05 2024-04-25T20:10:10.182 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.182 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:10.182 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.182 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.182 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.182 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:107 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'69, mlcod=16'64)", 2024-04-25T20:10:10.182 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:10:10.182 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 52.542138496, 2024-04-25T20:10:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028906270000000002, 2024-04-25T20:10:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:10:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:10:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:10.184 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584890+0000", 2024-04-25T20:10:10.184 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:10.184 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.184 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.184 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:10.184 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584895+0000", 2024-04-25T20:10:10.184 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2069999999999997e-06 2024-04-25T20:10:10.184 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.184 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.184 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:10.184 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584906+0000", 2024-04-25T20:10:10.184 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.117e-05 2024-04-25T20:10:10.184 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.184 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.184 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584910+0000", 2024-04-25T20:10:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4190000000000004e-06 2024-04-25T20:10:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585310+0000", 2024-04-25T20:10:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00040002700000000001 2024-04-25T20:10:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585329+0000", 2024-04-25T20:10:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8838000000000002e-05 2024-04-25T20:10:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587754+0000", 2024-04-25T20:10:10.186 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024252689999999999 2024-04-25T20:10:10.186 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.186 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.186 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:10.186 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587781+0000", 2024-04-25T20:10:10.186 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7206999999999999e-05 2024-04-25T20:10:10.186 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.186 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:10.186 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.186 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.186 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.186 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:108 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'70, mlcod=16'64)", 2024-04-25T20:10:10.186 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:10:10.186 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 52.542113344999997, 2024-04-25T20:10:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028924879999999999, 2024-04-25T20:10:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:10:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:10:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:10:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584919+0000", 2024-04-25T20:10:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.557e-06 2024-04-25T20:10:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584925+0000", 2024-04-25T20:10:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8619999999999999e-06 2024-04-25T20:10:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584931+0000", 2024-04-25T20:10:10.189 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2539999999999998e-06 2024-04-25T20:10:10.189 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.189 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.189 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:10.189 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585550+0000", 2024-04-25T20:10:10.189 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00061821800000000002 2024-04-25T20:10:10.189 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.189 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.189 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:10.189 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585567+0000", 2024-04-25T20:10:10.189 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7499e-05 2024-04-25T20:10:10.189 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.189 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.189 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:10.189 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587783+0000", 2024-04-25T20:10:10.189 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0022156200000000002 2024-04-25T20:10:10.190 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.190 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.190 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:10.190 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587808+0000", 2024-04-25T20:10:10.190 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5729e-05 2024-04-25T20:10:10.190 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.190 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:10.190 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.190 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.190 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.190 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:112 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'74, mlcod=16'66)", 2024-04-25T20:10:10.190 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:10:10.190 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 52.539955722999998, 2024-04-25T20:10:10.190 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00318724, 2024-04-25T20:10:10.190 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:10:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:10:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587073+0000", 2024-04-25T20:10:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587136+0000", 2024-04-25T20:10:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2928999999999993e-05 2024-04-25T20:10:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587147+0000", 2024-04-25T20:10:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1182999999999999e-05 2024-04-25T20:10:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587153+0000", 2024-04-25T20:10:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9059999999999999e-06 2024-04-25T20:10:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.193 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:10.193 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587306+0000", 2024-04-25T20:10:10.193 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000152848 2024-04-25T20:10:10.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.193 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.193 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:10.193 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587330+0000", 2024-04-25T20:10:10.193 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4304e-05 2024-04-25T20:10:10.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.193 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.193 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:10.193 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.590230+0000", 2024-04-25T20:10:10.193 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029004629999999998 2024-04-25T20:10:10.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.193 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.193 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:10.194 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.590261+0000", 2024-04-25T20:10:10.194 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0323000000000001e-05 2024-04-25T20:10:10.194 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.194 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:10.194 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.194 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.194 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.194 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:118 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'80, mlcod=16'78)", 2024-04-25T20:10:10.194 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:10:10.194 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 52.535169144000001, 2024-04-25T20:10:10.194 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029618959999999999, 2024-04-25T20:10:10.194 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:10.194 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:10.194 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:10.194 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.195 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:10.195 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:10:10.195 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.195 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.195 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.195 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:10.195 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:10:10.195 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.195 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.195 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.195 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:10.195 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591859+0000", 2024-04-25T20:10:10.195 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:10.195 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.195 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.196 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:10.196 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591918+0000", 2024-04-25T20:10:10.196 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8842999999999999e-05 2024-04-25T20:10:10.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.196 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.196 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:10.196 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591933+0000", 2024-04-25T20:10:10.196 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4600000000000001e-05 2024-04-25T20:10:10.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.196 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.196 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:10.196 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591939+0000", 2024-04-25T20:10:10.196 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4559999999999997e-06 2024-04-25T20:10:10.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.196 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591990+0000", 2024-04-25T20:10:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0318999999999997e-05 2024-04-25T20:10:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.592008+0000", 2024-04-25T20:10:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8819000000000001e-05 2024-04-25T20:10:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.594784+0000", 2024-04-25T20:10:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027750850000000001 2024-04-25T20:10:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:10.198 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.594822+0000", 2024-04-25T20:10:10.198 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8522000000000001e-05 2024-04-25T20:10:10.198 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.198 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:10.198 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.198 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.198 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:139 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'97, mlcod=17'87)", 2024-04-25T20:10:10.198 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:10:10.198 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 52.001132538999997, 2024-04-25T20:10:10.198 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048546290000000001, 2024-04-25T20:10:10.198 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:10.198 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:10.198 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:10:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:10:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125896+0000", 2024-04-25T20:10:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125908+0000", 2024-04-25T20:10:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.137e-05 2024-04-25T20:10:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125915+0000", 2024-04-25T20:10:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3939999999999998e-06 2024-04-25T20:10:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125922+0000", 2024-04-25T20:10:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1930000000000004e-06 2024-04-25T20:10:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125959+0000", 2024-04-25T20:10:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6701e-05 2024-04-25T20:10:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125972+0000", 2024-04-25T20:10:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2578000000000001e-05 2024-04-25T20:10:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130719+0000", 2024-04-25T20:10:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047475800000000004 2024-04-25T20:10:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130751+0000", 2024-04-25T20:10:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2283000000000002e-05 2024-04-25T20:10:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:140 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'98, mlcod=17'87)", 2024-04-25T20:10:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 52.000707188, 2024-04-25T20:10:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048310990000000002, 2024-04-25T20:10:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126427+0000", 2024-04-25T20:10:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000105572 2024-04-25T20:10:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126435+0000", 2024-04-25T20:10:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1920000000000005e-06 2024-04-25T20:10:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126439+0000", 2024-04-25T20:10:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6950000000000002e-06 2024-04-25T20:10:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126471+0000", 2024-04-25T20:10:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1699000000000003e-05 2024-04-25T20:10:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:10.205 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126484+0000", 2024-04-25T20:10:10.205 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.343e-05 2024-04-25T20:10:10.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.205 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.205 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:10.205 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131131+0000", 2024-04-25T20:10:10.205 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046471480000000003 2024-04-25T20:10:10.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.205 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.205 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:10.205 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131153+0000", 2024-04-25T20:10:10.205 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1753e-05 2024-04-25T20:10:10.205 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.205 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:10.205 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.206 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.206 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:141 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'99, mlcod=17'87)", 2024-04-25T20:10:10.206 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:10.206 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 52.000052244000003, 2024-04-25T20:10:10.206 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.004496847, 2024-04-25T20:10:10.206 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:10.206 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:10.206 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:10.206 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.206 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:10.206 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:10.206 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.206 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.206 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.206 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126980+0000", 2024-04-25T20:10:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.619e-06 2024-04-25T20:10:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126987+0000", 2024-04-25T20:10:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7220000000000002e-06 2024-04-25T20:10:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126992+0000", 2024-04-25T20:10:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8679999999999998e-06 2024-04-25T20:10:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130772+0000", 2024-04-25T20:10:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037803120000000001 2024-04-25T20:10:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130786+0000", 2024-04-25T20:10:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3592e-05 2024-04-25T20:10:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.209 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:10.209 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131451+0000", 2024-04-25T20:10:10.209 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000665206 2024-04-25T20:10:10.209 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:10.209 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:10.209 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:10.209 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131474+0000", 2024-04-25T20:10:10.209 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2957999999999999e-05 2024-04-25T20:10:10.209 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.209 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:10.209 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.209 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:10.209 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:10.209 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:10.211 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T20:10:10.211 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:10.210+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:10.312 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:10.312+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:10.357 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:10.358 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:10.358 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:10.358 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:10.367 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T20:10:10.413 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:10.413+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:10.487 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:10.487 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:10.487 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:10.488 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:10.488 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:10.497 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T20:10:10.541 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:10.541+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:10.603 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:10.603 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:10.603 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:10.603 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:10:10.603 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:10.612 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T20:10:10.653 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:10.653+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:10.715 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:10.715+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:10.759 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:10.759 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:10.759 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:10.759 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:10.768 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T20:10:10.816 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:10.816+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:10.874 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:10.874 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:10.874 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:10.875 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:10.875 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:10.883 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T20:10:10.927 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:10.927+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:10.991 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:10.992 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:10.992 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:10.992 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:10:10.992 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:11.001 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T20:10:11.045 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:11.045+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:11.114 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:11.114 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:11.114 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:10:11.114 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:11.119 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:11.119+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:11.123 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T20:10:11.220 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:11.220+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:11.265 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:11.265 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:11.265 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:11.265 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:11.265 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:11.274 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T20:10:11.321 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:11.321+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:11.375 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:11.375 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:11.375 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:11.375 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:10:11.375 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:11.383 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T20:10:11.402 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-25T20:10:11.426 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:11.427+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:11.485 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:11.485 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:11.485 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:10:11.485 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:11.493 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T20:10:11.523 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:11.523+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:11.624 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:11.624+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:11.635 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:11.635 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:11.635 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:11.635 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:11.635 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:11.644 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T20:10:11.725 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:11.725+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:11.784 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:11.785 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:11.785 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:11.785 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:10:11.785 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:11.793 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T20:10:11.826 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:11.825+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:11.896 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:11.896 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:11.896 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:10:11.896 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:11.905 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T20:10:11.922 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T20:10:11.966 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:11.966+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:12.006 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:12.007 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:12.007 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:12.007 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:12.007 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:12.015 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T20:10:12.028 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:12.028+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:12.062 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:10:12.063 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:10:12.080 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:10:12.080 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:10:12.096 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:10:12.097 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:10:12.114 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:10:12.114 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:10:12.116 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:12.116 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:12.116 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:12.116 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:10:12.117 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:12.126 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T20:10:12.129 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:12.128+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:12.131 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:10:12.131 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:10:12.162 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:10:12.162 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:10:12.178 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:10:12.179 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:10:12.195 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:10:12.195 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:10:12.228 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:12.228 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:12.237 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T20:10:12.251 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:12.251+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:12.329 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:12.329+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:12.339 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:12.339 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:12.348 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T20:10:12.410 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is unset 2024-04-25T20:10:12.430 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:12.430+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:12.450 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:12.450 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:12.458 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T20:10:12.532 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:12.531+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:12.602 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:12.602 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:12.614 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T20:10:12.633 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:12.633+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:12.733 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:12.733+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:12.781 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:12.781 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:12.790 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T20:10:12.833 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:12.834+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:12.934 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:12.934+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:12.991 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:12.991 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:13.000 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T20:10:13.042 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:13.042+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:13.136 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:13.137+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:13.157 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:13.157 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:13.169 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T20:10:13.237 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:13.237+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:13.279 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:13.279 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:13.288 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T20:10:13.338 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:13.338+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:13.389 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7] out_osds: [3, 2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-25T20:10:13.389 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-25T20:10:13.396 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:13.397 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:13.410 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T20:10:13.440 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:13.440+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:13.540 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:13.540+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:13.579 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:13.580 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:13.592 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T20:10:13.641 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:13.641+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:13.742 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:13.742+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:13.781 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:13.782 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:13.792 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T20:10:13.842 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:13.843+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:13.928 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:13.928 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:13.937 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T20:10:13.943 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:13.943+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:14.044 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:14.044+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:14.086 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:14.086 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:14.098 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T20:10:14.145 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:14.145+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:14.211 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T20:10:14.227 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:14.227 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:14.239 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T20:10:14.245 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:14.246+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:14.346 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:14.346+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:14.365 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:10:14.365 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:10:14.397 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:10:14.397 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:10:14.418 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.0 to 1 (10000) 2024-04-25T20:10:14.421 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:14.421 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:14.427 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-25T20:10:14.432 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:10:14.432 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:10:14.436 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T20:10:14.437 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-25T20:10:14.465 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:10:14.465 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:10:14.481 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:14.481+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:14.497 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:10:14.497 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:10:14.530 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:10:14.531 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:10:14.548 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:14.547+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:14.552 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:14.552 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:14.552 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:10:14.552 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:14.561 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T20:10:14.563 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:10:14.563 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:10:14.588 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:10:14.588 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:10:14.648 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:14.649+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:14.703 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:14.704 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:14.704 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:14.704 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:14.704 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:14.712 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T20:10:14.753 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:14.753+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:14.850 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:14.850+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:14.853 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:14.853 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:14.854 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:14.854 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:10:14.854 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:14.862 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T20:10:14.952 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:14.952+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:14.963 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:14.964 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:14.964 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:10:14.964 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:14.972 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T20:10:15.052 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:15.052+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:15.114 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:15.114 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:15.114 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:15.114 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:15.115 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:15.123 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T20:10:15.167 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:15.167+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:15.224 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:15.224 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:15.224 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:15.224 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:10:15.224 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:15.233 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T20:10:15.254 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:15.253+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:15.354 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:15.354+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:15.376 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:15.376 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:15.376 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:10:15.377 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:15.386 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T20:10:15.441 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is set 2024-04-25T20:10:15.441 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.1 to 1 (10000) 2024-04-25T20:10:15.455 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:15.455+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:15.461 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-04-25T20:10:15.489 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:15.489 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:15.489 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:15.490 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:15.490 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:15.498 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T20:10:15.555 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:15.555+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:15.641 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:15.642 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:15.642 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:15.642 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:10:15.642 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:15.650 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T20:10:15.695 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:15.695+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:15.752 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:15.752 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:15.752 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:10:15.752 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:15.756 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:15.757+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:15.760 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T20:10:15.857 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:15.858+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:15.902 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:15.902 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:15.903 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:15.903 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:15.903 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:15.912 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T20:10:15.996 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:15.996+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:16.056 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:16.056 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:16.056 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:16.056 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [ 2024-04-25T20:10:16.056 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.057 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:16.057 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:10:16.057 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 59.272650392999999, 2024-04-25T20:10:16.057 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.34154860100000001, 2024-04-25T20:10:16.057 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:16.057 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:16.057 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:16.057 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:16.057 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:16.057 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 1 2024-04-25T20:10:16.057 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.057 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:16.057 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.057 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:16.057 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:10:16.057 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.058 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.058 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.058 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:16.058 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:10:16.058 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.058 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.058 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.058 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:16.058 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782261+0000", 2024-04-25T20:10:16.058 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999986 2024-04-25T20:10:16.058 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.058 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.058 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:16.058 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782268+0000", 2024-04-25T20:10:16.058 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.8140000000000001e-06 2024-04-25T20:10:16.059 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.059 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.059 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:16.059 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782279+0000", 2024-04-25T20:10:16.059 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0665999999999999e-05 2024-04-25T20:10:16.059 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.059 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.059 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:16.059 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782290+0000", 2024-04-25T20:10:16.059 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.1343e-05 2024-04-25T20:10:16.059 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.059 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.059 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:16.059 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782340+0000", 2024-04-25T20:10:16.059 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.9311999999999998e-05 2024-04-25T20:10:16.059 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.060 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.060 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for peered", 2024-04-25T20:10:16.060 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782348+0000", 2024-04-25T20:10:16.060 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.0690000000000002e-06 2024-04-25T20:10:16.060 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.060 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.060 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:16.060 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120520+0000", 2024-04-25T20:10:16.060 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.33817188100000001 2024-04-25T20:10:16.060 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.060 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.060 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:16.060 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120656+0000", 2024-04-25T20:10:16.060 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000135897 2024-04-25T20:10:16.060 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.061 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.061 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:16.061 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120802+0000", 2024-04-25T20:10:16.061 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00014691199999999999 2024-04-25T20:10:16.061 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.061 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.061 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:16.061 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.122368+0000", 2024-04-25T20:10:16.061 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0015651040000000001 2024-04-25T20:10:16.061 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.061 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.061 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:16.061 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123729+0000", 2024-04-25T20:10:16.061 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0013610499999999999 2024-04-25T20:10:16.061 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.061 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.061 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:16.062 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123769+0000", 2024-04-25T20:10:16.062 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0719000000000001e-05 2024-04-25T20:10:16.062 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.062 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.062 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:16.062 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123811+0000", 2024-04-25T20:10:16.062 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1267999999999999e-05 2024-04-25T20:10:16.062 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.062 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:16.062 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.062 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.062 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.062 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:67 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:16.062 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:10:16.062 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 58.843231506000002, 2024-04-25T20:10:16.062 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.006947651, 2024-04-25T20:10:16.063 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:16.063 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:16.063 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:16.063 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:16.063 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:16.063 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 67 2024-04-25T20:10:16.063 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.063 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:16.063 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.063 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:16.063 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:10:16.063 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.063 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.063 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.063 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:16.063 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:10:16.064 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.064 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.064 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.064 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:16.064 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211680+0000", 2024-04-25T20:10:16.064 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:16.064 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.064 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.064 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:16.064 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211684+0000", 2024-04-25T20:10:16.064 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4249999999999998e-06 2024-04-25T20:10:16.064 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.064 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.064 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:16.064 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211690+0000", 2024-04-25T20:10:16.064 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.7360000000000004e-06 2024-04-25T20:10:16.065 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.065 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.065 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:16.065 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211694+0000", 2024-04-25T20:10:16.065 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4860000000000002e-06 2024-04-25T20:10:16.065 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.065 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.065 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:16.065 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211727+0000", 2024-04-25T20:10:16.065 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3445999999999998e-05 2024-04-25T20:10:16.065 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.065 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.065 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:16.065 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211778+0000", 2024-04-25T20:10:16.065 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1050000000000001e-05 2024-04-25T20:10:16.065 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.066 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.066 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:16.066 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211851+0000", 2024-04-25T20:10:16.066 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2914e-05 2024-04-25T20:10:16.066 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.066 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.066 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:16.066 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213824+0000", 2024-04-25T20:10:16.066 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0019726520000000001 2024-04-25T20:10:16.066 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.066 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.066 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:16.066 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218579+0000", 2024-04-25T20:10:16.066 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0047553159999999999 2024-04-25T20:10:16.067 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.067 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.067 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:16.067 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218607+0000", 2024-04-25T20:10:16.067 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8144999999999999e-05 2024-04-25T20:10:16.067 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.067 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.067 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:16.067 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218629+0000", 2024-04-25T20:10:16.067 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.1044e-05 2024-04-25T20:10:16.067 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.067 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:16.067 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.067 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.067 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.068 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:68 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:16.068 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:16.068 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 58.843213165000002, 2024-04-25T20:10:16.068 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012255596000000001, 2024-04-25T20:10:16.068 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:16.068 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:16.068 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:16.068 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:16.068 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:16.068 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 68 2024-04-25T20:10:16.068 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.068 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:16.068 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.068 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:16.068 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:16.069 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.069 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.069 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.069 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:16.069 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:16.069 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.069 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.069 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.069 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:16.069 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:16.069 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:16.069 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.069 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.069 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:16.069 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211701+0000", 2024-04-25T20:10:16.070 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2029999999999999e-06 2024-04-25T20:10:16.070 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.070 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.070 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:16.070 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211705+0000", 2024-04-25T20:10:16.070 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.2320000000000003e-06 2024-04-25T20:10:16.070 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.070 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.070 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:16.070 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211708+0000", 2024-04-25T20:10:16.070 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.6290000000000002e-06 2024-04-25T20:10:16.070 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.070 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.070 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:16.071 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212136+0000", 2024-04-25T20:10:16.071 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00042764099999999998 2024-04-25T20:10:16.071 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.071 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.071 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:16.071 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212212+0000", 2024-04-25T20:10:16.071 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.6315000000000004e-05 2024-04-25T20:10:16.071 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.071 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.071 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:16.071 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212306+0000", 2024-04-25T20:10:16.071 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.3556999999999999e-05 2024-04-25T20:10:16.071 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.071 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.072 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:16.072 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213829+0000", 2024-04-25T20:10:16.072 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001523256 2024-04-25T20:10:16.072 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.072 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.072 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:16.072 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223904+0000", 2024-04-25T20:10:16.072 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010075387 2024-04-25T20:10:16.072 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.072 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.072 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:16.072 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223934+0000", 2024-04-25T20:10:16.072 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9988000000000002e-05 2024-04-25T20:10:16.072 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.072 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.072 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:16.073 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223955+0000", 2024-04-25T20:10:16.073 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0630999999999999e-05 2024-04-25T20:10:16.073 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.073 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:16.073 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.073 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.073 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.073 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:69 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:16.073 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:16.073 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 58.843191429999997, 2024-04-25T20:10:16.073 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012316863000000001, 2024-04-25T20:10:16.073 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:16.073 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:16.073 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:16.073 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:16.073 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:16.074 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 69 2024-04-25T20:10:16.074 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.074 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:16.074 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.074 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:16.074 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:16.074 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.074 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.074 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.074 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:16.074 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:16.074 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.074 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.074 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.074 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:16.075 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:16.075 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:16.075 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.075 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.075 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:16.075 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211762+0000", 2024-04-25T20:10:16.075 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1703999999999999e-05 2024-04-25T20:10:16.075 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.075 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.075 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:16.075 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211768+0000", 2024-04-25T20:10:16.075 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.3430000000000003e-06 2024-04-25T20:10:16.075 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.075 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.075 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:16.076 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211771+0000", 2024-04-25T20:10:16.076 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.18e-06 2024-04-25T20:10:16.076 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.076 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.076 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:16.076 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212652+0000", 2024-04-25T20:10:16.076 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00088095800000000004 2024-04-25T20:10:16.076 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.076 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.076 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:16.076 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212703+0000", 2024-04-25T20:10:16.076 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.0655999999999997e-05 2024-04-25T20:10:16.076 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.076 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.076 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:16.076 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212781+0000", 2024-04-25T20:10:16.077 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.8852999999999999e-05 2024-04-25T20:10:16.077 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.077 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.077 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:16.077 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.214131+0000", 2024-04-25T20:10:16.077 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0013496039999999999 2024-04-25T20:10:16.077 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.077 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.077 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:16.077 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223986+0000", 2024-04-25T20:10:16.077 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.009854804 2024-04-25T20:10:16.077 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.077 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.077 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:16.077 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.224017+0000", 2024-04-25T20:10:16.077 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.1068999999999997e-05 2024-04-25T20:10:16.078 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.078 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.078 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:16.078 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.224038+0000", 2024-04-25T20:10:16.078 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0965e-05 2024-04-25T20:10:16.078 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.078 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:16.078 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.078 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.078 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.078 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:70 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:16.078 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:16.078 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 58.843136348999998, 2024-04-25T20:10:16.078 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.015776348999999999, 2024-04-25T20:10:16.078 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:16.078 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:16.079 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:16.079 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:16.079 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:16.079 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 70 2024-04-25T20:10:16.079 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.079 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:16.079 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.079 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:16.079 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:16.079 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.079 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.079 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.079 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:16.079 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:16.079 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.079 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.080 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.080 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:16.080 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:16.080 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:16.080 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.080 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.080 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:16.080 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211778+0000", 2024-04-25T20:10:16.080 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5330000000000001e-06 2024-04-25T20:10:16.080 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.080 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.080 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:16.080 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211782+0000", 2024-04-25T20:10:16.080 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4079999999999998e-06 2024-04-25T20:10:16.080 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.081 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.081 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:16.081 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211786+0000", 2024-04-25T20:10:16.081 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.6190000000000002e-06 2024-04-25T20:10:16.081 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.081 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.081 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:16.081 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213374+0000", 2024-04-25T20:10:16.081 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0015881859999999999 2024-04-25T20:10:16.081 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.081 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.081 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:16.081 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213478+0000", 2024-04-25T20:10:16.081 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000103464 2024-04-25T20:10:16.081 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.081 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.082 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:16.082 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213601+0000", 2024-04-25T20:10:16.082 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00012324300000000001 2024-04-25T20:10:16.082 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.082 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.082 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:16.082 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.214788+0000", 2024-04-25T20:10:16.082 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011868569999999999 2024-04-25T20:10:16.082 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.082 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.082 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:16.082 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227504+0000", 2024-04-25T20:10:16.082 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012715749 2024-04-25T20:10:16.082 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.082 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.082 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:16.083 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227532+0000", 2024-04-25T20:10:16.083 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.7937e-05 2024-04-25T20:10:16.083 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.083 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.083 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:16.083 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227552+0000", 2024-04-25T20:10:16.083 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0772999999999999e-05 2024-04-25T20:10:16.083 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.083 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:16.083 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.083 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.083 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.083 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:131 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:16.083 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:10:16.084 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 57.934126503999998, 2024-04-25T20:10:16.084 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0086158810000000006, 2024-04-25T20:10:16.084 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:16.084 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:16.084 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:16.084 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:16.084 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:16.084 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 131 2024-04-25T20:10:16.084 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.084 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:16.084 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.084 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:16.084 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:10:16.084 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.084 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.085 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.085 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:16.085 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:10:16.085 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.085 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.085 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.085 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:16.085 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120785+0000", 2024-04-25T20:10:16.085 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:16.085 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.085 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.085 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:16.085 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120880+0000", 2024-04-25T20:10:16.085 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.4925000000000003e-05 2024-04-25T20:10:16.085 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.086 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.086 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:16.086 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120891+0000", 2024-04-25T20:10:16.086 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0915e-05 2024-04-25T20:10:16.086 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.086 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.086 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:16.086 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120895+0000", 2024-04-25T20:10:16.086 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.8070000000000001e-06 2024-04-25T20:10:16.086 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.086 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.086 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:16.086 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121640+0000", 2024-04-25T20:10:16.086 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00074468500000000003 2024-04-25T20:10:16.086 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.086 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.087 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:16.087 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121684+0000", 2024-04-25T20:10:16.087 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4452000000000001e-05 2024-04-25T20:10:16.087 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.087 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.087 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:16.087 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121753+0000", 2024-04-25T20:10:16.087 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8508000000000002e-05 2024-04-25T20:10:16.087 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.087 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.087 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:16.087 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123618+0000", 2024-04-25T20:10:16.087 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0018655099999999999 2024-04-25T20:10:16.087 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.087 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.088 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:16.088 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129248+0000", 2024-04-25T20:10:16.088 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0056295540000000002 2024-04-25T20:10:16.088 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.088 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.088 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:16.088 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129368+0000", 2024-04-25T20:10:16.088 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000120244 2024-04-25T20:10:16.088 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.088 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.088 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:16.088 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129402+0000", 2024-04-25T20:10:16.088 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3812000000000001e-05 2024-04-25T20:10:16.088 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.088 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:16.088 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.089 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.089 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.089 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:132 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:16.089 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:16.089 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 57.934012293999999, 2024-04-25T20:10:16.089 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0085898169999999996, 2024-04-25T20:10:16.089 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:16.089 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:16.089 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:16.089 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:16.089 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:16.089 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 132 2024-04-25T20:10:16.089 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.089 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:16.089 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.089 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:16.090 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:16.090 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.090 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.090 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.090 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:16.090 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:16.090 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.090 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.090 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.090 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:16.090 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:16.090 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:16.090 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.090 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.090 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:16.090 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120902+0000", 2024-04-25T20:10:16.091 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.6129999999999998e-06 2024-04-25T20:10:16.091 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.091 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.091 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:16.091 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120906+0000", 2024-04-25T20:10:16.091 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6569999999999999e-06 2024-04-25T20:10:16.091 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.091 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.091 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:16.091 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120909+0000", 2024-04-25T20:10:16.091 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5550000000000001e-06 2024-04-25T20:10:16.091 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.091 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.091 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:16.092 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122098+0000", 2024-04-25T20:10:16.092 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011898049999999999 2024-04-25T20:10:16.092 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.092 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.092 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:16.092 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122172+0000", 2024-04-25T20:10:16.092 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3917000000000004e-05 2024-04-25T20:10:16.092 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.092 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.092 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:16.092 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122288+0000", 2024-04-25T20:10:16.092 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00011566 2024-04-25T20:10:16.092 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.092 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.092 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:16.093 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124519+0000", 2024-04-25T20:10:16.093 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0022308359999999999 2024-04-25T20:10:16.093 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.093 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.093 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:16.093 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129414+0000", 2024-04-25T20:10:16.093 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048953720000000003 2024-04-25T20:10:16.093 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.093 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.093 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:16.093 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129458+0000", 2024-04-25T20:10:16.093 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.3426000000000001e-05 2024-04-25T20:10:16.093 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.093 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.094 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:16.094 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129490+0000", 2024-04-25T20:10:16.094 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2329999999999997e-05 2024-04-25T20:10:16.094 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.094 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:16.094 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.094 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.094 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.094 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:133 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:16.094 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:16.094 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 57.933958425, 2024-04-25T20:10:16.094 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0086066270000000004, 2024-04-25T20:10:16.094 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:16.094 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:16.094 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:16.094 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:16.095 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:16.095 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 133 2024-04-25T20:10:16.095 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.095 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:16.095 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.095 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:16.095 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:16.095 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.095 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.095 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.095 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:16.095 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:16.095 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.095 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.095 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.096 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:16.096 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:16.096 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:16.096 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.096 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.096 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:16.096 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120956+0000", 2024-04-25T20:10:16.096 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5289999999999998e-06 2024-04-25T20:10:16.096 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.096 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.096 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:16.096 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120962+0000", 2024-04-25T20:10:16.096 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.9490000000000001e-06 2024-04-25T20:10:16.096 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.096 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.096 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:16.097 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120965+0000", 2024-04-25T20:10:16.097 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.1860000000000001e-06 2024-04-25T20:10:16.097 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.097 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.097 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:16.097 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122666+0000", 2024-04-25T20:10:16.097 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0017004500000000001 2024-04-25T20:10:16.097 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.097 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.097 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:16.097 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122741+0000", 2024-04-25T20:10:16.097 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.5297999999999996e-05 2024-04-25T20:10:16.097 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.097 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.097 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:16.097 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122858+0000", 2024-04-25T20:10:16.098 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00011723899999999999 2024-04-25T20:10:16.098 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.098 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.098 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:16.098 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124625+0000", 2024-04-25T20:10:16.098 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0017670100000000001 2024-04-25T20:10:16.098 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.098 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.098 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:16.098 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129497+0000", 2024-04-25T20:10:16.098 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048713580000000001 2024-04-25T20:10:16.098 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.098 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.099 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:16.099 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129532+0000", 2024-04-25T20:10:16.099 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5704999999999998e-05 2024-04-25T20:10:16.099 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.099 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.099 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:16.099 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129561+0000", 2024-04-25T20:10:16.099 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8283e-05 2024-04-25T20:10:16.099 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.099 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:16.099 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.099 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.099 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.099 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:134 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:16.099 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:16.099 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 57.933856227, 2024-04-25T20:10:16.100 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0085771179999999999, 2024-04-25T20:10:16.100 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:16.100 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:16.100 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:16.100 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:16.100 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:16.100 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 134 2024-04-25T20:10:16.100 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.100 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:16.100 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.100 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:16.100 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:16.100 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.100 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.100 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.100 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:16.100 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:16.101 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.101 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.101 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.101 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:16.101 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:16.101 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:16.101 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.101 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.101 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:16.101 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121157+0000", 2024-04-25T20:10:16.101 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00010104599999999999 2024-04-25T20:10:16.101 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.101 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.101 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:16.101 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121189+0000", 2024-04-25T20:10:16.102 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2589000000000003e-05 2024-04-25T20:10:16.102 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.102 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.102 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:16.102 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121195+0000", 2024-04-25T20:10:16.102 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.083e-06 2024-04-25T20:10:16.102 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.102 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.102 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:16.102 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123136+0000", 2024-04-25T20:10:16.102 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0019409780000000001 2024-04-25T20:10:16.102 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.102 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.102 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:16.102 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123181+0000", 2024-04-25T20:10:16.102 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4267999999999997e-05 2024-04-25T20:10:16.102 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.103 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.103 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:16.103 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123247+0000", 2024-04-25T20:10:16.103 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.6086000000000004e-05 2024-04-25T20:10:16.103 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.103 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.103 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:16.103 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125639+0000", 2024-04-25T20:10:16.103 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.002392 2024-04-25T20:10:16.103 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.103 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.103 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:16.103 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129567+0000", 2024-04-25T20:10:16.103 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0039282170000000003 2024-04-25T20:10:16.103 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.103 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.104 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:16.104 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129605+0000", 2024-04-25T20:10:16.104 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.7660999999999997e-05 2024-04-25T20:10:16.104 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.104 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.104 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:16.104 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129633+0000", 2024-04-25T20:10:16.104 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8649999999999998e-05 2024-04-25T20:10:16.104 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.104 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:16.104 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.104 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.104 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.104 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:135 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:16.104 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:10:16.104 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 57.933697221000003, 2024-04-25T20:10:16.105 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010898453000000001, 2024-04-25T20:10:16.105 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:16.105 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:16.105 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:16.105 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:16.105 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:16.105 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 135 2024-04-25T20:10:16.105 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.105 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:16.105 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.105 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:16.105 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:10:16.105 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.105 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.105 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.105 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:16.106 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:10:16.106 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.106 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.106 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.106 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:16.106 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121214+0000", 2024-04-25T20:10:16.106 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:16.106 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.106 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.106 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:16.106 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121218+0000", 2024-04-25T20:10:16.106 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6619999999999998e-06 2024-04-25T20:10:16.106 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.106 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.106 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:16.107 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121223+0000", 2024-04-25T20:10:16.107 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1070000000000004e-06 2024-04-25T20:10:16.107 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.107 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.107 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:16.107 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121227+0000", 2024-04-25T20:10:16.107 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2820000000000001e-06 2024-04-25T20:10:16.107 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.107 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.107 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:16.107 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123649+0000", 2024-04-25T20:10:16.107 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0024221970000000001 2024-04-25T20:10:16.107 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.107 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.107 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:16.107 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123723+0000", 2024-04-25T20:10:16.107 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.4386000000000002e-05 2024-04-25T20:10:16.108 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.108 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.108 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:16.108 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123844+0000", 2024-04-25T20:10:16.108 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000121249 2024-04-25T20:10:16.108 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.108 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.108 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:16.108 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125670+0000", 2024-04-25T20:10:16.108 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001825901 2024-04-25T20:10:16.108 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.108 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.108 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:16.108 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132043+0000", 2024-04-25T20:10:16.108 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0063724539999999996 2024-04-25T20:10:16.108 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.109 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.109 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:16.109 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132084+0000", 2024-04-25T20:10:16.109 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0963000000000001e-05 2024-04-25T20:10:16.109 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.109 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.109 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:16.109 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132114+0000", 2024-04-25T20:10:16.109 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9927e-05 2024-04-25T20:10:16.109 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.109 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:16.109 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.109 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.109 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.109 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:136 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:16.109 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:10:16.110 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 57.933681694999997, 2024-04-25T20:10:16.110 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010952513000000001, 2024-04-25T20:10:16.110 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:16.110 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:16.110 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:16.110 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:16.110 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:16.110 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 136 2024-04-25T20:10:16.110 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.110 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:16.110 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.110 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:16.110 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:10:16.110 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.110 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.110 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.110 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:16.111 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:10:16.111 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.111 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.111 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.111 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:16.111 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121230+0000", 2024-04-25T20:10:16.111 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:16.111 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.111 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.111 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:16.111 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121236+0000", 2024-04-25T20:10:16.111 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.7320000000000003e-06 2024-04-25T20:10:16.111 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.111 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.111 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:16.112 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121242+0000", 2024-04-25T20:10:16.112 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.7810000000000002e-06 2024-04-25T20:10:16.112 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.112 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.112 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:16.112 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121246+0000", 2024-04-25T20:10:16.112 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.639e-06 2024-04-25T20:10:16.112 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.112 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.112 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:16.112 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124150+0000", 2024-04-25T20:10:16.112 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.002903816 2024-04-25T20:10:16.112 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.112 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.112 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:16.112 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124195+0000", 2024-04-25T20:10:16.112 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.5042e-05 2024-04-25T20:10:16.113 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.113 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.113 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:16.113 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124266+0000", 2024-04-25T20:10:16.113 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0234999999999994e-05 2024-04-25T20:10:16.113 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.113 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.113 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:16.113 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125711+0000", 2024-04-25T20:10:16.113 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0014458800000000001 2024-04-25T20:10:16.113 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.113 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.113 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:16.113 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132120+0000", 2024-04-25T20:10:16.113 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0064084399999999996 2024-04-25T20:10:16.113 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.114 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.114 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:16.114 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132155+0000", 2024-04-25T20:10:16.114 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4938000000000003e-05 2024-04-25T20:10:16.114 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.114 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.114 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:16.114 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132183+0000", 2024-04-25T20:10:16.114 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8360999999999998e-05 2024-04-25T20:10:16.114 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.114 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:16.114 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.114 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.114 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.114 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:16.114 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:16.115 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 57.933631963000003, 2024-04-25T20:10:16.115 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.01097684, 2024-04-25T20:10:16.115 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:16.115 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:16.115 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:16.115 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:16.115 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:16.115 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 137 2024-04-25T20:10:16.115 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.115 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:16.115 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.115 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:16.115 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:16.115 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.115 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.115 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.115 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:16.116 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:16.116 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.116 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.116 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.116 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:16.116 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:16.116 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:16.116 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.116 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.116 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:16.116 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121328+0000", 2024-04-25T20:10:16.116 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.7908999999999998e-05 2024-04-25T20:10:16.116 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.116 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.116 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:16.116 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121337+0000", 2024-04-25T20:10:16.116 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.9369999999999993e-06 2024-04-25T20:10:16.117 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.117 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.117 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:16.117 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121343+0000", 2024-04-25T20:10:16.117 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.8089999999999998e-06 2024-04-25T20:10:16.117 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.117 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.117 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:16.117 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124656+0000", 2024-04-25T20:10:16.117 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0033133839999999999 2024-04-25T20:10:16.117 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.117 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.117 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:16.117 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124730+0000", 2024-04-25T20:10:16.117 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3565000000000005e-05 2024-04-25T20:10:16.117 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.118 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.118 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:16.118 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124846+0000", 2024-04-25T20:10:16.118 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000116416 2024-04-25T20:10:16.118 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.118 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.118 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:16.118 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126756+0000", 2024-04-25T20:10:16.118 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001909991 2024-04-25T20:10:16.118 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.118 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.118 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:16.118 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132190+0000", 2024-04-25T20:10:16.118 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0054343880000000001 2024-04-25T20:10:16.118 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.118 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.119 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:16.119 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132227+0000", 2024-04-25T20:10:16.119 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6875000000000002e-05 2024-04-25T20:10:16.119 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.119 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.119 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:16.119 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132257+0000", 2024-04-25T20:10:16.119 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.0046e-05 2024-04-25T20:10:16.119 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.119 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:16.119 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.119 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.119 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.119 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:16.119 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:16.119 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 57.933562004999999, 2024-04-25T20:10:16.119 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010978653999999999, 2024-04-25T20:10:16.120 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:16.120 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:16.120 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:16.120 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:16.120 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:16.120 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 138 2024-04-25T20:10:16.120 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.120 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:16.120 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.120 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:16.120 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:16.120 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.120 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.120 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.120 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:16.120 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:16.121 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.121 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.121 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.121 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:16.121 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:16.121 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:16.121 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.121 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.121 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:16.121 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121355+0000", 2024-04-25T20:10:16.121 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.9869999999999999e-06 2024-04-25T20:10:16.121 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.121 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.121 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:16.121 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121361+0000", 2024-04-25T20:10:16.121 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.4620000000000001e-06 2024-04-25T20:10:16.122 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.122 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.122 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:16.122 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121373+0000", 2024-04-25T20:10:16.122 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.2035000000000001e-05 2024-04-25T20:10:16.122 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.122 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.122 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:16.122 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125267+0000", 2024-04-25T20:10:16.122 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0038939230000000001 2024-04-25T20:10:16.122 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.122 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.122 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:16.122 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125309+0000", 2024-04-25T20:10:16.122 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1690999999999998e-05 2024-04-25T20:10:16.122 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.123 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.123 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:16.123 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125385+0000", 2024-04-25T20:10:16.123 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.6162999999999996e-05 2024-04-25T20:10:16.123 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.123 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.123 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:16.123 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126798+0000", 2024-04-25T20:10:16.123 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0014124680000000001 2024-04-25T20:10:16.123 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.123 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.123 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:16.123 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132264+0000", 2024-04-25T20:10:16.123 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0054661689999999999 2024-04-25T20:10:16.123 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.123 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.124 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:16.124 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132299+0000", 2024-04-25T20:10:16.124 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5272999999999999e-05 2024-04-25T20:10:16.124 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.124 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.124 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:16.124 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132329+0000", 2024-04-25T20:10:16.124 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.0080999999999999e-05 2024-04-25T20:10:16.124 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.124 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:16.124 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.124 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.124 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.124 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:16.124 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:10:16.124 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 57.933531430000002, 2024-04-25T20:10:16.124 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011015853000000001, 2024-04-25T20:10:16.125 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:16.125 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:16.125 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:16.125 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:16.125 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:16.125 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 139 2024-04-25T20:10:16.125 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.125 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:16.125 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.125 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:16.125 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:10:16.125 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.125 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.125 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.125 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:16.125 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:10:16.126 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.126 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.126 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.126 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:16.126 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121380+0000", 2024-04-25T20:10:16.126 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:16.126 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.126 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.126 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:16.126 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121385+0000", 2024-04-25T20:10:16.126 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.092e-06 2024-04-25T20:10:16.126 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.126 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.126 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:16.126 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121390+0000", 2024-04-25T20:10:16.126 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.5990000000000001e-06 2024-04-25T20:10:16.127 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.127 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.127 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:16.127 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121397+0000", 2024-04-25T20:10:16.127 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.6279999999999999e-06 2024-04-25T20:10:16.127 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.127 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.127 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:16.127 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125737+0000", 2024-04-25T20:10:16.127 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0043406010000000004 2024-04-25T20:10:16.127 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.127 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.127 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:16.127 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125781+0000", 2024-04-25T20:10:16.127 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.3958999999999997e-05 2024-04-25T20:10:16.127 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.127 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.128 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:16.128 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125849+0000", 2024-04-25T20:10:16.128 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8071999999999995e-05 2024-04-25T20:10:16.128 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.128 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.128 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:16.128 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.130908+0000", 2024-04-25T20:10:16.128 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0050586509999999999 2024-04-25T20:10:16.128 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.128 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.128 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:16.128 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132334+0000", 2024-04-25T20:10:16.128 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00142621 2024-04-25T20:10:16.128 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.128 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.128 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:16.129 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132369+0000", 2024-04-25T20:10:16.129 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4452000000000002e-05 2024-04-25T20:10:16.129 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.129 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.129 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:16.129 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132397+0000", 2024-04-25T20:10:16.129 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8065e-05 2024-04-25T20:10:16.129 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.129 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:16.129 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.129 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.129 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.129 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:16.129 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:16.129 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 57.933510386000002, 2024-04-25T20:10:16.130 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011069443, 2024-04-25T20:10:16.130 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:16.130 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:16.130 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:16.130 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:16.130 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:16.130 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 140 2024-04-25T20:10:16.130 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.130 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:16.130 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.130 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:16.130 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:16.130 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.130 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.130 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.130 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:16.131 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:16.131 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.131 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.131 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.131 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:16.131 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:16.131 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:16.131 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.131 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.131 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:16.131 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121456+0000", 2024-04-25T20:10:16.131 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.4469000000000003e-05 2024-04-25T20:10:16.131 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.131 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.131 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:16.131 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121462+0000", 2024-04-25T20:10:16.131 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.1890000000000004e-06 2024-04-25T20:10:16.132 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.132 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.132 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:16.132 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121466+0000", 2024-04-25T20:10:16.132 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3909999999999998e-06 2024-04-25T20:10:16.132 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.132 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.132 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:16.132 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126071+0000", 2024-04-25T20:10:16.132 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0046048979999999996 2024-04-25T20:10:16.132 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.132 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.132 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:16.132 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126141+0000", 2024-04-25T20:10:16.132 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0621e-05 2024-04-25T20:10:16.133 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.133 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.133 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:16.133 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126260+0000", 2024-04-25T20:10:16.133 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000118621 2024-04-25T20:10:16.133 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.133 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.133 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:16.133 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.131270+0000", 2024-04-25T20:10:16.133 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0050096949999999998 2024-04-25T20:10:16.133 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.133 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.133 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:16.133 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132403+0000", 2024-04-25T20:10:16.133 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011332779999999999 2024-04-25T20:10:16.133 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.134 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.134 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:16.134 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132443+0000", 2024-04-25T20:10:16.134 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0175000000000001e-05 2024-04-25T20:10:16.134 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.134 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.134 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:16.134 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132471+0000", 2024-04-25T20:10:16.134 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8399e-05 2024-04-25T20:10:16.134 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.134 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:16.134 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.134 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.134 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.134 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:16.134 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:10:16.135 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 57.933441600999998, 2024-04-25T20:10:16.135 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011071002, 2024-04-25T20:10:16.135 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:16.135 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:16.135 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:16.135 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:16.135 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:16.135 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 141 2024-04-25T20:10:16.135 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.135 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:16.135 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.135 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:16.135 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:10:16.135 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.135 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.135 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.136 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:16.136 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:10:16.136 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.136 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.136 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.136 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:16.136 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121470+0000", 2024-04-25T20:10:16.136 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:16.136 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.136 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.136 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:16.136 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121473+0000", 2024-04-25T20:10:16.136 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.01e-06 2024-04-25T20:10:16.136 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.136 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.136 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:16.136 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121478+0000", 2024-04-25T20:10:16.137 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.515e-06 2024-04-25T20:10:16.137 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.137 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.137 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:16.137 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121480+0000", 2024-04-25T20:10:16.137 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5749999999999999e-06 2024-04-25T20:10:16.137 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.137 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.137 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:16.137 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126813+0000", 2024-04-25T20:10:16.137 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0053320399999999997 2024-04-25T20:10:16.137 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.137 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.137 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:16.138 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126854+0000", 2024-04-25T20:10:16.138 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1582999999999998e-05 2024-04-25T20:10:16.138 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.138 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.138 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:16.138 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126923+0000", 2024-04-25T20:10:16.138 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8544e-05 2024-04-25T20:10:16.138 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.138 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.138 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:16.138 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.131573+0000", 2024-04-25T20:10:16.138 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0046504659999999998 2024-04-25T20:10:16.138 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.138 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.139 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:16.139 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132478+0000", 2024-04-25T20:10:16.139 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00090442400000000003 2024-04-25T20:10:16.139 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.139 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.139 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:16.139 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132513+0000", 2024-04-25T20:10:16.139 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5204999999999999e-05 2024-04-25T20:10:16.139 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.139 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.139 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:16.139 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132542+0000", 2024-04-25T20:10:16.139 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.904e-05 2024-04-25T20:10:16.139 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.139 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:16.139 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.140 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.140 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.140 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:156 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:16.140 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:16.140 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 57.435617305999997, 2024-04-25T20:10:16.140 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090072280000000008, 2024-04-25T20:10:16.140 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:16.140 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:16.140 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:16.140 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:16.140 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:16.140 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 156 2024-04-25T20:10:16.140 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.140 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:16.140 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.140 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:16.140 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:16.141 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.141 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.141 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.141 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:16.141 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:16.141 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.141 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.141 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.141 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:16.141 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:16.141 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:16.141 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.141 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.141 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:16.141 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619297+0000", 2024-04-25T20:10:16.141 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2979999999999999e-06 2024-04-25T20:10:16.142 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.142 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.142 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:16.142 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619300+0000", 2024-04-25T20:10:16.142 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.371e-06 2024-04-25T20:10:16.142 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.142 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.142 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:16.142 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619304+0000", 2024-04-25T20:10:16.142 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.304e-06 2024-04-25T20:10:16.142 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.142 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.142 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:16.142 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.622910+0000", 2024-04-25T20:10:16.142 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0036065049999999999 2024-04-25T20:10:16.143 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.143 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.143 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:16.143 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.622990+0000", 2024-04-25T20:10:16.143 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.9722999999999995e-05 2024-04-25T20:10:16.143 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.143 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.143 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:16.143 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623128+0000", 2024-04-25T20:10:16.143 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00013818399999999999 2024-04-25T20:10:16.143 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.143 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.143 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:16.143 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625155+0000", 2024-04-25T20:10:16.143 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0020267979999999998 2024-04-25T20:10:16.143 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.143 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.144 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:16.144 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628225+0000", 2024-04-25T20:10:16.144 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0030705680000000001 2024-04-25T20:10:16.144 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.144 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.144 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:16.144 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628268+0000", 2024-04-25T20:10:16.144 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.2404000000000003e-05 2024-04-25T20:10:16.144 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.144 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.144 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:16.144 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628302+0000", 2024-04-25T20:10:16.144 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4439e-05 2024-04-25T20:10:16.144 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.144 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:16.145 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.145 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.145 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.145 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:157 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:16.145 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:16.145 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 57.435605207000002, 2024-04-25T20:10:16.145 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090534889999999996, 2024-04-25T20:10:16.145 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:16.145 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:16.145 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:16.145 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:16.145 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:16.145 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 157 2024-04-25T20:10:16.145 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.145 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:16.145 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.146 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:16.146 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:16.146 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.146 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.146 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.146 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:16.146 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:16.146 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.146 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.146 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.146 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:16.146 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:16.146 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:16.146 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.146 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.147 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:16.147 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619309+0000", 2024-04-25T20:10:16.147 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.841e-06 2024-04-25T20:10:16.147 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.147 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.147 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:16.147 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619312+0000", 2024-04-25T20:10:16.147 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.084e-06 2024-04-25T20:10:16.147 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.147 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.147 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:16.147 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619314+0000", 2024-04-25T20:10:16.147 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.3860000000000001e-06 2024-04-25T20:10:16.147 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.147 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.147 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:16.148 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623718+0000", 2024-04-25T20:10:16.148 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0044038050000000002 2024-04-25T20:10:16.148 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.148 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.148 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:16.148 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623791+0000", 2024-04-25T20:10:16.148 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3181000000000003e-05 2024-04-25T20:10:16.148 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.148 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.148 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:16.148 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623922+0000", 2024-04-25T20:10:16.148 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00013117000000000001 2024-04-25T20:10:16.148 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.148 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.148 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:16.148 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625187+0000", 2024-04-25T20:10:16.148 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001264961 2024-04-25T20:10:16.149 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.149 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.149 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:16.149 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628308+0000", 2024-04-25T20:10:16.149 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.003120739 2024-04-25T20:10:16.149 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.149 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.149 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:16.149 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628331+0000", 2024-04-25T20:10:16.149 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2802999999999998e-05 2024-04-25T20:10:16.149 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.149 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.149 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:16.149 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628361+0000", 2024-04-25T20:10:16.149 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9760999999999998e-05 2024-04-25T20:10:16.149 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.150 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:16.150 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.150 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.150 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.150 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:158 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:16.150 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:10:16.150 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 57.435594760000001, 2024-04-25T20:10:16.150 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0091076409999999997, 2024-04-25T20:10:16.150 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:16.150 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:16.150 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:16.150 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:16.150 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:16.150 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 158 2024-04-25T20:10:16.150 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.150 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:16.151 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.151 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:16.151 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:10:16.151 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.151 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.151 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.151 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:16.151 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:10:16.151 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.151 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.151 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.151 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:16.151 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619317+0000", 2024-04-25T20:10:16.151 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967296 2024-04-25T20:10:16.151 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.151 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.152 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:16.152 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619364+0000", 2024-04-25T20:10:16.152 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.6572000000000001e-05 2024-04-25T20:10:16.152 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.152 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.152 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:16.152 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619371+0000", 2024-04-25T20:10:16.152 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3860000000000001e-06 2024-04-25T20:10:16.152 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.152 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.152 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:16.152 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619382+0000", 2024-04-25T20:10:16.152 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0173999999999999e-05 2024-04-25T20:10:16.152 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.152 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.152 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:16.153 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624216+0000", 2024-04-25T20:10:16.153 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048340350000000004 2024-04-25T20:10:16.153 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.153 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.153 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:16.153 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624260+0000", 2024-04-25T20:10:16.153 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4558999999999998e-05 2024-04-25T20:10:16.153 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.153 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.153 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:16.153 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624333+0000", 2024-04-25T20:10:16.153 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2744999999999996e-05 2024-04-25T20:10:16.153 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.153 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.153 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:16.153 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625465+0000", 2024-04-25T20:10:16.154 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001132454 2024-04-25T20:10:16.154 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.154 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.154 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:16.154 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628365+0000", 2024-04-25T20:10:16.154 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0028994950000000002 2024-04-25T20:10:16.154 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.154 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.154 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:16.154 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628386+0000", 2024-04-25T20:10:16.154 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.1367000000000001e-05 2024-04-25T20:10:16.154 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.154 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.154 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:16.154 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628425+0000", 2024-04-25T20:10:16.154 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.9087e-05 2024-04-25T20:10:16.155 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.155 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:16.155 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.155 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.155 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.155 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:159 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:16.155 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:16.155 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 57.435526107000001, 2024-04-25T20:10:16.155 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090948999999999995, 2024-04-25T20:10:16.155 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:16.155 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:16.155 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:16.155 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:16.155 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:16.155 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 159 2024-04-25T20:10:16.155 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.156 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:16.156 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.156 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:16.156 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:16.156 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.156 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.156 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.156 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:16.156 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:16.156 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:16.156 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.156 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.156 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:16.156 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:16.156 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:16.157 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.157 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.157 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:16.157 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619389+0000", 2024-04-25T20:10:16.157 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.734e-06 2024-04-25T20:10:16.157 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.157 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.157 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:16.157 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619392+0000", 2024-04-25T20:10:16.157 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6219999999999998e-06 2024-04-25T20:10:16.157 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.157 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.157 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:16.157 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619403+0000", 2024-04-25T20:10:16.157 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0987e-05 2024-04-25T20:10:16.158 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.158 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.158 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:16.158 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624667+0000", 2024-04-25T20:10:16.158 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0052633970000000004 2024-04-25T20:10:16.158 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.158 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.158 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:16.158 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624741+0000", 2024-04-25T20:10:16.158 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.4583000000000001e-05 2024-04-25T20:10:16.158 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.158 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.158 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:16.158 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624857+0000", 2024-04-25T20:10:16.159 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000116118 2024-04-25T20:10:16.159 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.159 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.159 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:16.159 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.626013+0000", 2024-04-25T20:10:16.159 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001155473 2024-04-25T20:10:16.159 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.159 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.159 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:16.159 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628432+0000", 2024-04-25T20:10:16.159 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0024189379999999998 2024-04-25T20:10:16.159 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.159 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.159 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:16.159 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628454+0000", 2024-04-25T20:10:16.159 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2059999999999999e-05 2024-04-25T20:10:16.160 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:16.160 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:16.160 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:16.160 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628481+0000", 2024-04-25T20:10:16.160 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.7300999999999998e-05 2024-04-25T20:10:16.160 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.160 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:16.160 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.160 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:16.160 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:16.160 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:16.161 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:16.162+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:16.162 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T20:10:16.262 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:16.263+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:16.293 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:16.293 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:16.293 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:16.293 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:16.302 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T20:10:16.363 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:16.363+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:16.424 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:16.424 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:16.424 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:16.424 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:16.424 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:16.436 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T20:10:16.451 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.4 to 1 (10000) 2024-04-25T20:10:16.463 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:16.462+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:16.469 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-04-25T20:10:16.564 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:16.565+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:16.569 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:16.569 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:16.569 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:16.569 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:10:16.569 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:16.581 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T20:10:16.605 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T20:10:16.665 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:16.665+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:16.744 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:16.744 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:16.744 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:16.744 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:16.756 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T20:10:16.800 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:10:16.800 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:10:16.801 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:16.799+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:16.826 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:10:16.827 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:10:16.859 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:10:16.859 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:10:16.867 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:16.867+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:16.897 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:10:16.897 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:10:16.925 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:16.925 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:16.929 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:10:16.929 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:10:16.937 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T20:10:16.961 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:10:16.961 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:10:16.981 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:10:16.982 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:10:16.982 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:16.980+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:17.013 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:10:17.013 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:10:17.068 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:17.067+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:17.105 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:17.105 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:17.117 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T20:10:17.201 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:17.200+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:17.269 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:17.268+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:17.304 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:17.304 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:17.316 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T20:10:17.370 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:17.370+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:17.462 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-25T20:10:17.471 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.5 to 1 (10000) 2024-04-25T20:10:17.472 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:17.470+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:17.475 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:17.475 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:17.490 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T20:10:17.491 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-04-25T20:10:17.571 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:17.572+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:17.672 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:17.673+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:17.686 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:17.686 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:17.697 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T20:10:17.773 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:17.773+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:17.852 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:17.852 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:17.865 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T20:10:17.910 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:17.909+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:17.975 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:17.975+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:18.015 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:18.015 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:18.028 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T20:10:18.076 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:18.075+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:18.177 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:18.177+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:18.239 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:18.239 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:18.252 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T20:10:18.277 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:18.276+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:18.366 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:18.366 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:18.376 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T20:10:18.377 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:18.376+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:18.466 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.6 to 1 (10000) 2024-04-25T20:10:18.467 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is set 2024-04-25T20:10:18.478 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:18.477+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:18.478 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:18.482 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:18.485 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-04-25T20:10:18.488 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T20:10:18.579 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:18.579+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:18.589 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:18.590 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:18.598 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T20:10:18.680 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:18.679+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:18.740 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:18.740 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:18.749 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T20:10:18.781 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:18.780+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:18.850 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:18.850 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:18.858 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T20:10:18.902 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:18.902+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:18.959 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:18.960 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:18.968 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T20:10:18.982 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:18.981+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:19.026 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T20:10:19.083 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:19.083+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:19.110 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:19.110 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:19.119 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T20:10:19.200 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:10:19.200 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:10:19.203 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:19.203+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:19.218 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:10:19.219 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:10:19.236 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:10:19.236 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:10:19.253 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:10:19.254 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:10:19.260 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:19.260 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:19.269 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T20:10:19.273 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:10:19.273 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:10:19.286 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:19.284+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:19.304 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:10:19.305 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:10:19.334 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:10:19.335 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:10:19.353 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:10:19.353 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:10:19.408 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:19.407+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:19.413 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:19.413 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:19.413 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:19.413 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:10:19.413 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:19.421 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T20:10:19.471 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.7 to 1 (10000) 2024-04-25T20:10:19.486 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:19.487+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:19.488 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-25T20:10:19.488 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T20:10:19.524 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:19.524 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:19.525 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:10:19.525 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:19.533 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T20:10:19.617 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:19.617+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:19.675 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:19.675 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:19.675 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:19.676 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:19.676 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:19.684 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T20:10:19.725 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:19.725+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:19.789 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:19.788+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:19.827 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:19.827 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:19.827 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:19.827 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [ 2024-04-25T20:10:19.827 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.827 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:19.827 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:10:19.827 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 63.043421250999998, 2024-04-25T20:10:19.827 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.34154860100000001, 2024-04-25T20:10:19.827 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:19.828 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:19.828 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:19.828 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:19.828 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:19.828 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 1 2024-04-25T20:10:19.828 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.828 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:19.828 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.828 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:19.828 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:10:19.828 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.828 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.828 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.828 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:19.828 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:10:19.828 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.829 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.829 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.829 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:19.829 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782261+0000", 2024-04-25T20:10:19.829 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999986 2024-04-25T20:10:19.829 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.829 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.829 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:19.829 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782268+0000", 2024-04-25T20:10:19.829 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.8140000000000001e-06 2024-04-25T20:10:19.829 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.829 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.829 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:19.829 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782279+0000", 2024-04-25T20:10:19.830 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0665999999999999e-05 2024-04-25T20:10:19.830 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.830 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.830 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:19.830 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782290+0000", 2024-04-25T20:10:19.830 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.1343e-05 2024-04-25T20:10:19.830 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.830 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.830 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:19.830 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782340+0000", 2024-04-25T20:10:19.830 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.9311999999999998e-05 2024-04-25T20:10:19.830 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.830 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.830 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for peered", 2024-04-25T20:10:19.830 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782348+0000", 2024-04-25T20:10:19.830 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.0690000000000002e-06 2024-04-25T20:10:19.831 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.831 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.831 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:19.831 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120520+0000", 2024-04-25T20:10:19.831 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.33817188100000001 2024-04-25T20:10:19.831 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.831 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.831 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:19.831 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120656+0000", 2024-04-25T20:10:19.831 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000135897 2024-04-25T20:10:19.831 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.831 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.831 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:19.831 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120802+0000", 2024-04-25T20:10:19.831 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00014691199999999999 2024-04-25T20:10:19.831 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.832 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.832 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:19.832 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.122368+0000", 2024-04-25T20:10:19.832 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0015651040000000001 2024-04-25T20:10:19.832 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.832 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.832 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:19.832 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123729+0000", 2024-04-25T20:10:19.832 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0013610499999999999 2024-04-25T20:10:19.832 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.832 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.832 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:19.832 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123769+0000", 2024-04-25T20:10:19.832 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0719000000000001e-05 2024-04-25T20:10:19.832 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.832 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.833 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:19.833 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123811+0000", 2024-04-25T20:10:19.833 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1267999999999999e-05 2024-04-25T20:10:19.833 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.833 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:19.833 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.833 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.833 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.833 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:67 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:19.833 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:10:19.833 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 62.614002364000001, 2024-04-25T20:10:19.833 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.006947651, 2024-04-25T20:10:19.833 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:19.833 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:19.833 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:19.833 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:19.834 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:19.834 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 67 2024-04-25T20:10:19.834 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.834 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:19.834 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.834 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:19.834 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:10:19.834 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.834 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.834 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.834 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:19.834 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:10:19.834 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.834 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.834 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.835 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:19.835 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211680+0000", 2024-04-25T20:10:19.835 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:19.835 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.835 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.835 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:19.835 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211684+0000", 2024-04-25T20:10:19.835 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4249999999999998e-06 2024-04-25T20:10:19.835 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.835 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.835 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:19.835 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211690+0000", 2024-04-25T20:10:19.835 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.7360000000000004e-06 2024-04-25T20:10:19.835 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.835 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.835 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:19.836 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211694+0000", 2024-04-25T20:10:19.836 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4860000000000002e-06 2024-04-25T20:10:19.836 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.836 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.836 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:19.836 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211727+0000", 2024-04-25T20:10:19.836 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3445999999999998e-05 2024-04-25T20:10:19.836 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.836 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.836 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:19.836 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211778+0000", 2024-04-25T20:10:19.836 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1050000000000001e-05 2024-04-25T20:10:19.836 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.836 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.836 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:19.836 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211851+0000", 2024-04-25T20:10:19.836 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2914e-05 2024-04-25T20:10:19.837 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.837 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.837 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:19.837 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213824+0000", 2024-04-25T20:10:19.837 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0019726520000000001 2024-04-25T20:10:19.837 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.837 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.837 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:19.837 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218579+0000", 2024-04-25T20:10:19.837 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0047553159999999999 2024-04-25T20:10:19.837 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.837 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.837 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:19.837 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218607+0000", 2024-04-25T20:10:19.838 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8144999999999999e-05 2024-04-25T20:10:19.838 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.838 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.838 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:19.838 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218629+0000", 2024-04-25T20:10:19.838 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.1044e-05 2024-04-25T20:10:19.838 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.838 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:19.838 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.838 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.838 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.838 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:68 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:19.838 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:19.838 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 62.613984023, 2024-04-25T20:10:19.839 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012255596000000001, 2024-04-25T20:10:19.839 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:19.839 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:19.839 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:19.839 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:19.839 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:19.839 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 68 2024-04-25T20:10:19.839 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.839 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:19.839 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.839 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:19.839 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:19.839 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.840 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.840 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.840 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:19.840 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:19.840 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.840 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.840 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.840 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:19.840 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:19.840 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:19.840 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.840 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.840 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:19.841 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211701+0000", 2024-04-25T20:10:19.841 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2029999999999999e-06 2024-04-25T20:10:19.841 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.841 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.841 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:19.841 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211705+0000", 2024-04-25T20:10:19.841 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.2320000000000003e-06 2024-04-25T20:10:19.841 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.841 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.841 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:19.841 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211708+0000", 2024-04-25T20:10:19.841 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.6290000000000002e-06 2024-04-25T20:10:19.841 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.841 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.841 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:19.841 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212136+0000", 2024-04-25T20:10:19.842 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00042764099999999998 2024-04-25T20:10:19.842 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.842 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.842 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:19.842 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212212+0000", 2024-04-25T20:10:19.842 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.6315000000000004e-05 2024-04-25T20:10:19.842 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.842 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.842 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:19.842 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212306+0000", 2024-04-25T20:10:19.842 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.3556999999999999e-05 2024-04-25T20:10:19.842 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.842 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.842 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:19.842 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213829+0000", 2024-04-25T20:10:19.843 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001523256 2024-04-25T20:10:19.843 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.843 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.843 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:19.843 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223904+0000", 2024-04-25T20:10:19.843 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010075387 2024-04-25T20:10:19.843 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.843 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.843 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:19.843 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223934+0000", 2024-04-25T20:10:19.843 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9988000000000002e-05 2024-04-25T20:10:19.843 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.843 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.843 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:19.843 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223955+0000", 2024-04-25T20:10:19.843 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0630999999999999e-05 2024-04-25T20:10:19.843 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.844 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:19.844 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.844 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.844 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.844 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:69 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:19.844 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:19.844 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 62.613962288000003, 2024-04-25T20:10:19.844 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012316863000000001, 2024-04-25T20:10:19.844 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:19.844 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:19.844 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:19.844 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:19.844 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:19.844 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 69 2024-04-25T20:10:19.844 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.844 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:19.845 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.845 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:19.845 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:19.845 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.845 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.845 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.845 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:19.845 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:19.845 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.845 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.845 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.845 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:19.845 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:19.846 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:19.846 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.846 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.846 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:19.846 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211762+0000", 2024-04-25T20:10:19.846 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1703999999999999e-05 2024-04-25T20:10:19.846 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.846 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.846 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:19.846 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211768+0000", 2024-04-25T20:10:19.846 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.3430000000000003e-06 2024-04-25T20:10:19.847 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.847 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.847 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:19.847 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211771+0000", 2024-04-25T20:10:19.847 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.18e-06 2024-04-25T20:10:19.847 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.847 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.847 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:19.847 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212652+0000", 2024-04-25T20:10:19.847 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00088095800000000004 2024-04-25T20:10:19.847 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.847 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.847 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:19.847 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212703+0000", 2024-04-25T20:10:19.847 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.0655999999999997e-05 2024-04-25T20:10:19.847 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.847 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.848 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:19.848 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212781+0000", 2024-04-25T20:10:19.848 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.8852999999999999e-05 2024-04-25T20:10:19.848 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.848 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.848 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:19.848 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.214131+0000", 2024-04-25T20:10:19.848 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0013496039999999999 2024-04-25T20:10:19.848 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.848 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.848 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:19.848 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223986+0000", 2024-04-25T20:10:19.848 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.009854804 2024-04-25T20:10:19.849 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.849 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.849 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:19.849 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.224017+0000", 2024-04-25T20:10:19.849 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.1068999999999997e-05 2024-04-25T20:10:19.849 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.849 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.849 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:19.849 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.224038+0000", 2024-04-25T20:10:19.849 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0965e-05 2024-04-25T20:10:19.849 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.849 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:19.849 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.849 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.849 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.849 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:70 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:19.850 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:19.850 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 62.613907206999997, 2024-04-25T20:10:19.850 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.015776348999999999, 2024-04-25T20:10:19.850 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:19.850 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:19.850 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:19.850 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:19.850 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:19.850 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 70 2024-04-25T20:10:19.850 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.850 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:19.850 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.850 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:19.850 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:19.850 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.850 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.851 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.851 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:19.851 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:19.851 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.851 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.851 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.851 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:19.851 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:19.851 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:19.851 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.851 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.851 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:19.851 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211778+0000", 2024-04-25T20:10:19.851 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5330000000000001e-06 2024-04-25T20:10:19.851 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.851 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.851 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:19.851 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211782+0000", 2024-04-25T20:10:19.852 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4079999999999998e-06 2024-04-25T20:10:19.852 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.852 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.852 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:19.852 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211786+0000", 2024-04-25T20:10:19.852 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.6190000000000002e-06 2024-04-25T20:10:19.852 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.852 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.852 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:19.852 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213374+0000", 2024-04-25T20:10:19.852 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0015881859999999999 2024-04-25T20:10:19.852 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.852 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.853 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:19.853 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213478+0000", 2024-04-25T20:10:19.853 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000103464 2024-04-25T20:10:19.853 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.853 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.853 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:19.853 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213601+0000", 2024-04-25T20:10:19.853 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00012324300000000001 2024-04-25T20:10:19.853 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.853 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.853 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:19.853 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.214788+0000", 2024-04-25T20:10:19.853 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011868569999999999 2024-04-25T20:10:19.854 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.854 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.854 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:19.854 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227504+0000", 2024-04-25T20:10:19.854 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012715749 2024-04-25T20:10:19.854 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.854 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.854 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:19.854 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227532+0000", 2024-04-25T20:10:19.854 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.7937e-05 2024-04-25T20:10:19.854 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.854 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.854 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:19.854 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227552+0000", 2024-04-25T20:10:19.855 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0772999999999999e-05 2024-04-25T20:10:19.855 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.855 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:19.855 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.855 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.855 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.855 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:131 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:19.855 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:10:19.855 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 61.704897361999997, 2024-04-25T20:10:19.855 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0086158810000000006, 2024-04-25T20:10:19.855 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:19.855 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:19.855 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:19.855 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:19.855 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:19.856 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 131 2024-04-25T20:10:19.856 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.856 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:19.856 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.856 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:19.856 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:10:19.856 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.856 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.856 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.856 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:19.856 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:10:19.856 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.856 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.856 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.856 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:19.856 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120785+0000", 2024-04-25T20:10:19.857 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:19.857 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.857 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.857 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:19.857 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120880+0000", 2024-04-25T20:10:19.857 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.4925000000000003e-05 2024-04-25T20:10:19.857 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.857 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.857 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:19.857 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120891+0000", 2024-04-25T20:10:19.857 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0915e-05 2024-04-25T20:10:19.857 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.857 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.857 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:19.857 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120895+0000", 2024-04-25T20:10:19.857 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.8070000000000001e-06 2024-04-25T20:10:19.858 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.858 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.858 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:19.858 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121640+0000", 2024-04-25T20:10:19.858 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00074468500000000003 2024-04-25T20:10:19.858 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.858 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.858 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:19.858 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121684+0000", 2024-04-25T20:10:19.858 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4452000000000001e-05 2024-04-25T20:10:19.858 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.858 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.858 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:19.858 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121753+0000", 2024-04-25T20:10:19.859 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8508000000000002e-05 2024-04-25T20:10:19.859 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.859 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.859 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:19.859 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123618+0000", 2024-04-25T20:10:19.859 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0018655099999999999 2024-04-25T20:10:19.859 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.859 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.859 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:19.859 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129248+0000", 2024-04-25T20:10:19.860 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0056295540000000002 2024-04-25T20:10:19.860 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.860 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.860 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:19.860 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129368+0000", 2024-04-25T20:10:19.860 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000120244 2024-04-25T20:10:19.860 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.860 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.860 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:19.860 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129402+0000", 2024-04-25T20:10:19.861 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3812000000000001e-05 2024-04-25T20:10:19.861 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.861 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:19.861 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.861 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.861 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.861 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:132 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:19.861 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:19.861 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 61.704783151999997, 2024-04-25T20:10:19.861 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0085898169999999996, 2024-04-25T20:10:19.862 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:19.862 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:19.862 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:19.862 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:19.862 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:19.862 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 132 2024-04-25T20:10:19.862 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.862 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:19.862 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.862 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:19.862 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:19.863 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.863 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.863 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.863 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:19.863 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:19.863 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.863 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.863 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.863 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:19.863 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:19.863 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:19.864 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.864 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.864 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:19.864 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120902+0000", 2024-04-25T20:10:19.864 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.6129999999999998e-06 2024-04-25T20:10:19.864 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.864 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.864 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:19.864 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120906+0000", 2024-04-25T20:10:19.864 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6569999999999999e-06 2024-04-25T20:10:19.865 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.865 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.865 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:19.865 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120909+0000", 2024-04-25T20:10:19.865 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5550000000000001e-06 2024-04-25T20:10:19.865 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.865 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.865 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:19.865 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122098+0000", 2024-04-25T20:10:19.865 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011898049999999999 2024-04-25T20:10:19.865 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.865 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.865 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:19.866 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122172+0000", 2024-04-25T20:10:19.866 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3917000000000004e-05 2024-04-25T20:10:19.866 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.866 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.866 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:19.866 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122288+0000", 2024-04-25T20:10:19.866 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00011566 2024-04-25T20:10:19.866 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.866 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.866 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:19.866 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124519+0000", 2024-04-25T20:10:19.866 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0022308359999999999 2024-04-25T20:10:19.866 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.866 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.866 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:19.867 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129414+0000", 2024-04-25T20:10:19.867 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048953720000000003 2024-04-25T20:10:19.867 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.867 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.867 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:19.867 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129458+0000", 2024-04-25T20:10:19.867 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.3426000000000001e-05 2024-04-25T20:10:19.867 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.867 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.867 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:19.867 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129490+0000", 2024-04-25T20:10:19.867 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2329999999999997e-05 2024-04-25T20:10:19.867 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.867 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:19.868 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.868 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.868 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.868 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:133 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:19.868 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:19.868 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 61.704729282999999, 2024-04-25T20:10:19.868 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0086066270000000004, 2024-04-25T20:10:19.868 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:19.868 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:19.868 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:19.868 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:19.868 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:19.869 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 133 2024-04-25T20:10:19.869 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.869 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:19.869 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.869 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:19.869 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:19.869 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.869 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.869 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.869 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:19.870 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:19.870 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.870 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.870 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.870 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:19.870 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:19.870 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:19.870 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.870 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.871 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:19.871 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120956+0000", 2024-04-25T20:10:19.871 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5289999999999998e-06 2024-04-25T20:10:19.871 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.871 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.871 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:19.871 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120962+0000", 2024-04-25T20:10:19.871 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.9490000000000001e-06 2024-04-25T20:10:19.871 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.871 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.872 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:19.872 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120965+0000", 2024-04-25T20:10:19.872 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.1860000000000001e-06 2024-04-25T20:10:19.872 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.872 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.872 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:19.872 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122666+0000", 2024-04-25T20:10:19.872 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0017004500000000001 2024-04-25T20:10:19.872 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.872 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.872 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:19.873 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122741+0000", 2024-04-25T20:10:19.873 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.5297999999999996e-05 2024-04-25T20:10:19.873 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.873 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.873 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:19.873 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122858+0000", 2024-04-25T20:10:19.873 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00011723899999999999 2024-04-25T20:10:19.873 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.873 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.873 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:19.874 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124625+0000", 2024-04-25T20:10:19.874 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0017670100000000001 2024-04-25T20:10:19.874 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.874 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.874 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:19.874 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129497+0000", 2024-04-25T20:10:19.874 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048713580000000001 2024-04-25T20:10:19.874 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.874 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.874 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:19.874 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129532+0000", 2024-04-25T20:10:19.874 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5704999999999998e-05 2024-04-25T20:10:19.875 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.875 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.875 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:19.875 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129561+0000", 2024-04-25T20:10:19.875 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8283e-05 2024-04-25T20:10:19.875 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.875 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:19.875 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.875 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.875 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.875 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:134 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:19.875 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:19.875 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 61.704627084999998, 2024-04-25T20:10:19.875 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0085771179999999999, 2024-04-25T20:10:19.875 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:19.876 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:19.876 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:19.876 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:19.876 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:19.876 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 134 2024-04-25T20:10:19.876 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.876 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:19.876 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.876 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:19.876 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:19.876 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.876 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.876 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.876 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:19.876 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:19.877 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.877 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.877 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.877 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:19.877 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:19.877 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:19.877 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.877 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.877 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:19.877 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121157+0000", 2024-04-25T20:10:19.877 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00010104599999999999 2024-04-25T20:10:19.877 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.877 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.877 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:19.877 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121189+0000", 2024-04-25T20:10:19.877 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2589000000000003e-05 2024-04-25T20:10:19.878 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.878 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.878 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:19.878 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121195+0000", 2024-04-25T20:10:19.878 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.083e-06 2024-04-25T20:10:19.878 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.878 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.878 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:19.878 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123136+0000", 2024-04-25T20:10:19.878 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0019409780000000001 2024-04-25T20:10:19.878 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.878 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.878 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:19.878 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123181+0000", 2024-04-25T20:10:19.878 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4267999999999997e-05 2024-04-25T20:10:19.879 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.879 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.879 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:19.879 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123247+0000", 2024-04-25T20:10:19.879 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.6086000000000004e-05 2024-04-25T20:10:19.879 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.879 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.879 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:19.879 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125639+0000", 2024-04-25T20:10:19.879 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.002392 2024-04-25T20:10:19.879 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.879 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.879 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:19.879 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129567+0000", 2024-04-25T20:10:19.879 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0039282170000000003 2024-04-25T20:10:19.880 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.880 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.880 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:19.880 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129605+0000", 2024-04-25T20:10:19.880 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.7660999999999997e-05 2024-04-25T20:10:19.880 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.880 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.880 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:19.880 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129633+0000", 2024-04-25T20:10:19.880 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8649999999999998e-05 2024-04-25T20:10:19.880 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.880 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:19.880 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.880 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.880 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.880 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:135 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:19.881 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:10:19.881 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 61.704468079000002, 2024-04-25T20:10:19.881 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010898453000000001, 2024-04-25T20:10:19.881 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:19.881 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:19.881 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:19.881 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:19.881 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:19.881 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 135 2024-04-25T20:10:19.881 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.881 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:19.881 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.881 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:19.881 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:10:19.881 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.882 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.882 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.882 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:19.882 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:10:19.882 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.882 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.882 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.882 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:19.882 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121214+0000", 2024-04-25T20:10:19.882 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:19.882 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.882 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.882 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:19.882 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121218+0000", 2024-04-25T20:10:19.882 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6619999999999998e-06 2024-04-25T20:10:19.883 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.883 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.883 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:19.883 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121223+0000", 2024-04-25T20:10:19.883 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1070000000000004e-06 2024-04-25T20:10:19.883 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.883 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.883 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:19.883 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121227+0000", 2024-04-25T20:10:19.883 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2820000000000001e-06 2024-04-25T20:10:19.883 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.883 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.883 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:19.883 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123649+0000", 2024-04-25T20:10:19.883 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0024221970000000001 2024-04-25T20:10:19.884 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.884 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.884 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:19.884 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123723+0000", 2024-04-25T20:10:19.884 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.4386000000000002e-05 2024-04-25T20:10:19.884 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.884 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.884 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:19.884 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123844+0000", 2024-04-25T20:10:19.884 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000121249 2024-04-25T20:10:19.884 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.884 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.884 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:19.884 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125670+0000", 2024-04-25T20:10:19.884 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001825901 2024-04-25T20:10:19.884 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.885 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.885 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:19.885 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132043+0000", 2024-04-25T20:10:19.885 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0063724539999999996 2024-04-25T20:10:19.885 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.885 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.885 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:19.885 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132084+0000", 2024-04-25T20:10:19.885 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0963000000000001e-05 2024-04-25T20:10:19.885 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.885 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.885 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:19.885 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132114+0000", 2024-04-25T20:10:19.885 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9927e-05 2024-04-25T20:10:19.885 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.886 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:19.886 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.886 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.886 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.886 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:136 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:19.886 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:10:19.886 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 61.704452553000003, 2024-04-25T20:10:19.886 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010952513000000001, 2024-04-25T20:10:19.886 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:19.886 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:19.886 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:19.886 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:19.886 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:19.886 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 136 2024-04-25T20:10:19.887 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.887 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:19.887 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.887 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:19.887 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:10:19.887 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.887 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.887 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.887 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:19.887 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:10:19.887 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.887 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.887 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.887 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:19.887 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121230+0000", 2024-04-25T20:10:19.887 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:19.888 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.888 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.888 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:19.888 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121236+0000", 2024-04-25T20:10:19.888 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.7320000000000003e-06 2024-04-25T20:10:19.888 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.888 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.888 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:19.888 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121242+0000", 2024-04-25T20:10:19.888 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.7810000000000002e-06 2024-04-25T20:10:19.888 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.888 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.888 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:19.888 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121246+0000", 2024-04-25T20:10:19.888 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.639e-06 2024-04-25T20:10:19.889 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.889 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.889 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:19.889 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124150+0000", 2024-04-25T20:10:19.889 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.002903816 2024-04-25T20:10:19.889 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.889 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.889 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:19.889 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124195+0000", 2024-04-25T20:10:19.889 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.5042e-05 2024-04-25T20:10:19.889 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.889 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.889 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:19.889 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124266+0000", 2024-04-25T20:10:19.890 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0234999999999994e-05 2024-04-25T20:10:19.890 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.890 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.890 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:19.890 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125711+0000", 2024-04-25T20:10:19.890 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0014458800000000001 2024-04-25T20:10:19.890 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.890 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.890 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:19.890 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132120+0000", 2024-04-25T20:10:19.890 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0064084399999999996 2024-04-25T20:10:19.890 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.890 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.890 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:19.890 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132155+0000", 2024-04-25T20:10:19.890 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4938000000000003e-05 2024-04-25T20:10:19.891 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.891 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.891 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:19.891 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132183+0000", 2024-04-25T20:10:19.891 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8360999999999998e-05 2024-04-25T20:10:19.891 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.891 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:19.891 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.891 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.891 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.891 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:19.891 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:19.891 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 61.704402821000002, 2024-04-25T20:10:19.891 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.01097684, 2024-04-25T20:10:19.891 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:19.891 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:19.892 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:19.892 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:19.892 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:19.892 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 137 2024-04-25T20:10:19.892 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.892 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:19.892 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.892 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:19.892 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:19.892 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.892 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.892 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.892 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:19.892 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:19.892 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.892 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.892 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.893 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:19.893 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:19.893 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:19.893 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.893 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.893 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:19.893 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121328+0000", 2024-04-25T20:10:19.893 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.7908999999999998e-05 2024-04-25T20:10:19.893 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.893 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.893 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:19.893 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121337+0000", 2024-04-25T20:10:19.893 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.9369999999999993e-06 2024-04-25T20:10:19.893 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.893 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.894 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:19.894 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121343+0000", 2024-04-25T20:10:19.894 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.8089999999999998e-06 2024-04-25T20:10:19.894 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.894 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.894 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:19.894 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124656+0000", 2024-04-25T20:10:19.894 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0033133839999999999 2024-04-25T20:10:19.894 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.894 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.894 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:19.894 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124730+0000", 2024-04-25T20:10:19.894 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3565000000000005e-05 2024-04-25T20:10:19.895 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.895 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.895 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:19.895 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124846+0000", 2024-04-25T20:10:19.895 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000116416 2024-04-25T20:10:19.895 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.895 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.895 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:19.895 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126756+0000", 2024-04-25T20:10:19.895 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001909991 2024-04-25T20:10:19.895 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.895 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.895 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:19.895 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132190+0000", 2024-04-25T20:10:19.895 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0054343880000000001 2024-04-25T20:10:19.895 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.896 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.896 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:19.896 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132227+0000", 2024-04-25T20:10:19.896 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6875000000000002e-05 2024-04-25T20:10:19.896 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.896 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.896 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:19.896 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132257+0000", 2024-04-25T20:10:19.896 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.0046e-05 2024-04-25T20:10:19.896 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.896 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:19.896 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.896 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.896 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.896 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:19.896 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:19.897 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 61.704332862999998, 2024-04-25T20:10:19.897 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010978653999999999, 2024-04-25T20:10:19.897 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:19.897 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:19.897 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:19.897 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:19.897 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:19.897 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 138 2024-04-25T20:10:19.897 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.897 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:19.897 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.897 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:19.897 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:19.897 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.897 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.898 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.898 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:19.898 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:19.898 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.898 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.898 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.898 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:19.898 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:19.898 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:19.898 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.898 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.898 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:19.898 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121355+0000", 2024-04-25T20:10:19.898 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.9869999999999999e-06 2024-04-25T20:10:19.898 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.899 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.899 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:19.899 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121361+0000", 2024-04-25T20:10:19.899 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.4620000000000001e-06 2024-04-25T20:10:19.899 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.899 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.899 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:19.899 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121373+0000", 2024-04-25T20:10:19.899 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.2035000000000001e-05 2024-04-25T20:10:19.899 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.899 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.899 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:19.899 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125267+0000", 2024-04-25T20:10:19.899 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0038939230000000001 2024-04-25T20:10:19.900 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.900 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.900 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:19.900 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125309+0000", 2024-04-25T20:10:19.900 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1690999999999998e-05 2024-04-25T20:10:19.900 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.900 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.900 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:19.900 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125385+0000", 2024-04-25T20:10:19.900 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.6162999999999996e-05 2024-04-25T20:10:19.900 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.900 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.900 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:19.900 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126798+0000", 2024-04-25T20:10:19.900 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0014124680000000001 2024-04-25T20:10:19.901 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.901 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.901 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:19.901 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132264+0000", 2024-04-25T20:10:19.901 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0054661689999999999 2024-04-25T20:10:19.901 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.901 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.901 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:19.901 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132299+0000", 2024-04-25T20:10:19.901 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5272999999999999e-05 2024-04-25T20:10:19.901 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.901 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.901 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:19.901 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132329+0000", 2024-04-25T20:10:19.901 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.0080999999999999e-05 2024-04-25T20:10:19.901 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.902 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:19.902 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.902 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.902 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.902 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:19.902 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:10:19.902 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 61.704302288000001, 2024-04-25T20:10:19.902 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011015853000000001, 2024-04-25T20:10:19.902 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:19.902 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:19.902 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:19.902 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:19.902 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:19.902 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 139 2024-04-25T20:10:19.902 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.903 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:19.903 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.903 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:19.903 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:10:19.903 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.903 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.903 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.903 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:19.903 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:10:19.903 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.903 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.903 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.903 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:19.903 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121380+0000", 2024-04-25T20:10:19.903 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:19.903 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.903 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.904 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:19.904 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121385+0000", 2024-04-25T20:10:19.904 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.092e-06 2024-04-25T20:10:19.904 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.904 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.904 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:19.904 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121390+0000", 2024-04-25T20:10:19.904 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.5990000000000001e-06 2024-04-25T20:10:19.904 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.904 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.904 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:19.904 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121397+0000", 2024-04-25T20:10:19.904 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.6279999999999999e-06 2024-04-25T20:10:19.904 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.904 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.905 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:19.905 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125737+0000", 2024-04-25T20:10:19.905 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0043406010000000004 2024-04-25T20:10:19.905 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.905 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.905 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:19.905 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125781+0000", 2024-04-25T20:10:19.905 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.3958999999999997e-05 2024-04-25T20:10:19.905 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.905 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.905 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:19.905 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125849+0000", 2024-04-25T20:10:19.905 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8071999999999995e-05 2024-04-25T20:10:19.905 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.905 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.905 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:19.906 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.130908+0000", 2024-04-25T20:10:19.906 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0050586509999999999 2024-04-25T20:10:19.906 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.906 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.906 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:19.906 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132334+0000", 2024-04-25T20:10:19.906 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00142621 2024-04-25T20:10:19.906 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.906 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.906 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:19.906 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132369+0000", 2024-04-25T20:10:19.906 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4452000000000002e-05 2024-04-25T20:10:19.906 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.906 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.906 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:19.907 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132397+0000", 2024-04-25T20:10:19.907 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8065e-05 2024-04-25T20:10:19.907 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.907 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:19.907 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.907 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.907 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.907 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:19.907 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:19.907 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 61.704281244000001, 2024-04-25T20:10:19.907 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011069443, 2024-04-25T20:10:19.907 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:19.907 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:19.908 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:19.908 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:19.908 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:19.908 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 140 2024-04-25T20:10:19.908 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.908 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:19.908 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.908 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:19.908 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:19.908 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.908 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.908 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.908 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:19.908 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:19.908 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.909 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.909 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.909 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:19.909 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:19.909 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:19.909 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.909 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.909 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:19.909 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121456+0000", 2024-04-25T20:10:19.909 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.4469000000000003e-05 2024-04-25T20:10:19.909 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.909 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.909 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:19.909 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121462+0000", 2024-04-25T20:10:19.909 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.1890000000000004e-06 2024-04-25T20:10:19.910 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.910 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.910 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:19.910 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121466+0000", 2024-04-25T20:10:19.910 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3909999999999998e-06 2024-04-25T20:10:19.910 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.910 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.910 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:19.910 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126071+0000", 2024-04-25T20:10:19.910 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0046048979999999996 2024-04-25T20:10:19.910 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.910 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.910 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:19.910 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126141+0000", 2024-04-25T20:10:19.910 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0621e-05 2024-04-25T20:10:19.910 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.910 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.911 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:19.911 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126260+0000", 2024-04-25T20:10:19.911 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000118621 2024-04-25T20:10:19.911 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.911 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.911 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:19.911 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.131270+0000", 2024-04-25T20:10:19.911 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0050096949999999998 2024-04-25T20:10:19.911 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.911 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.911 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:19.911 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132403+0000", 2024-04-25T20:10:19.911 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011332779999999999 2024-04-25T20:10:19.911 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.911 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.912 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:19.912 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132443+0000", 2024-04-25T20:10:19.912 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0175000000000001e-05 2024-04-25T20:10:19.912 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.912 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.912 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:19.912 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132471+0000", 2024-04-25T20:10:19.912 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8399e-05 2024-04-25T20:10:19.912 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.912 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:19.912 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.912 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.912 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.912 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:19.912 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:10:19.912 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 61.704212458999997, 2024-04-25T20:10:19.913 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011071002, 2024-04-25T20:10:19.913 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:19.913 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:19.913 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:19.913 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:19.913 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:19.913 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 141 2024-04-25T20:10:19.913 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.913 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:19.913 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.913 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:19.913 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:10:19.913 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.913 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.913 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.914 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:19.914 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:10:19.914 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.914 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.914 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.914 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:19.914 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121470+0000", 2024-04-25T20:10:19.914 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:19.914 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.914 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.914 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:19.914 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121473+0000", 2024-04-25T20:10:19.914 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.01e-06 2024-04-25T20:10:19.914 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.914 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.914 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:19.915 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121478+0000", 2024-04-25T20:10:19.915 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.515e-06 2024-04-25T20:10:19.915 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.915 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.915 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:19.915 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121480+0000", 2024-04-25T20:10:19.915 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5749999999999999e-06 2024-04-25T20:10:19.915 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.915 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.915 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:19.915 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126813+0000", 2024-04-25T20:10:19.915 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0053320399999999997 2024-04-25T20:10:19.915 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.915 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.915 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:19.915 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126854+0000", 2024-04-25T20:10:19.915 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1582999999999998e-05 2024-04-25T20:10:19.916 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.917 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.917 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:19.917 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126923+0000", 2024-04-25T20:10:19.917 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8544e-05 2024-04-25T20:10:19.917 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.917 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.917 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:19.917 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.131573+0000", 2024-04-25T20:10:19.917 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0046504659999999998 2024-04-25T20:10:19.918 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.918 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.918 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:19.918 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132478+0000", 2024-04-25T20:10:19.918 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00090442400000000003 2024-04-25T20:10:19.918 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.918 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.918 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:19.918 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132513+0000", 2024-04-25T20:10:19.918 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5204999999999999e-05 2024-04-25T20:10:19.918 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.919 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.919 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:19.919 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132542+0000", 2024-04-25T20:10:19.919 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.904e-05 2024-04-25T20:10:19.919 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.919 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:19.919 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.919 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.919 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.919 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:156 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:19.919 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:19.919 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 61.206388164000003, 2024-04-25T20:10:19.919 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090072280000000008, 2024-04-25T20:10:19.920 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:19.920 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:19.920 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:19.920 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:19.920 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:19.920 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 156 2024-04-25T20:10:19.920 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.920 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:19.920 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.920 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:19.920 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:19.920 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.920 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.920 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.920 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:19.921 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:19.921 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.921 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.921 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.921 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:19.921 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:19.921 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:19.921 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.921 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.921 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:19.921 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619297+0000", 2024-04-25T20:10:19.921 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2979999999999999e-06 2024-04-25T20:10:19.921 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.921 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.921 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:19.921 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619300+0000", 2024-04-25T20:10:19.921 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.371e-06 2024-04-25T20:10:19.922 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.922 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.922 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:19.922 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619304+0000", 2024-04-25T20:10:19.922 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.304e-06 2024-04-25T20:10:19.922 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.922 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.922 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:19.922 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.622910+0000", 2024-04-25T20:10:19.922 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0036065049999999999 2024-04-25T20:10:19.922 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.922 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.922 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:19.922 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.622990+0000", 2024-04-25T20:10:19.922 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.9722999999999995e-05 2024-04-25T20:10:19.922 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.923 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.923 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:19.923 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623128+0000", 2024-04-25T20:10:19.923 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00013818399999999999 2024-04-25T20:10:19.923 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.923 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.923 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:19.923 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625155+0000", 2024-04-25T20:10:19.923 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0020267979999999998 2024-04-25T20:10:19.923 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.923 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.923 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:19.923 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628225+0000", 2024-04-25T20:10:19.923 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0030705680000000001 2024-04-25T20:10:19.923 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.923 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.923 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:19.924 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628268+0000", 2024-04-25T20:10:19.924 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.2404000000000003e-05 2024-04-25T20:10:19.924 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.924 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.924 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:19.924 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628302+0000", 2024-04-25T20:10:19.924 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4439e-05 2024-04-25T20:10:19.924 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.924 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:19.924 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.924 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.924 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.924 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:157 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:19.924 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:19.924 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 61.206376065000001, 2024-04-25T20:10:19.924 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090534889999999996, 2024-04-25T20:10:19.924 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:19.925 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:19.925 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:19.925 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:19.925 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:19.925 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 157 2024-04-25T20:10:19.925 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.925 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:19.925 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.925 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:19.925 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:19.925 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.925 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.925 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.925 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:19.925 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:19.925 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.925 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.926 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.926 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:19.926 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:19.926 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:19.926 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.926 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.926 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:19.926 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619309+0000", 2024-04-25T20:10:19.926 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.841e-06 2024-04-25T20:10:19.926 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.926 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.926 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:19.926 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619312+0000", 2024-04-25T20:10:19.926 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.084e-06 2024-04-25T20:10:19.926 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.927 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.927 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:19.927 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619314+0000", 2024-04-25T20:10:19.927 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.3860000000000001e-06 2024-04-25T20:10:19.927 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.927 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.927 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:19.927 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623718+0000", 2024-04-25T20:10:19.927 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0044038050000000002 2024-04-25T20:10:19.927 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.927 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.927 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:19.927 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623791+0000", 2024-04-25T20:10:19.927 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3181000000000003e-05 2024-04-25T20:10:19.927 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.927 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.927 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:19.928 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623922+0000", 2024-04-25T20:10:19.928 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00013117000000000001 2024-04-25T20:10:19.928 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.928 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.928 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:19.928 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625187+0000", 2024-04-25T20:10:19.928 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001264961 2024-04-25T20:10:19.928 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.928 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.928 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:19.928 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628308+0000", 2024-04-25T20:10:19.928 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.003120739 2024-04-25T20:10:19.928 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.928 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.928 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:19.929 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628331+0000", 2024-04-25T20:10:19.929 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2802999999999998e-05 2024-04-25T20:10:19.929 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.929 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.929 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:19.929 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628361+0000", 2024-04-25T20:10:19.929 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9760999999999998e-05 2024-04-25T20:10:19.929 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.929 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:19.929 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.929 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.929 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.929 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:158 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:19.929 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:10:19.929 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 61.206365618, 2024-04-25T20:10:19.929 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0091076409999999997, 2024-04-25T20:10:19.930 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:19.930 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:19.930 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:19.930 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:19.930 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:19.930 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 158 2024-04-25T20:10:19.930 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.930 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:19.930 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.930 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:19.930 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:10:19.930 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.930 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.930 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.931 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:19.931 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:10:19.931 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.931 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.931 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.931 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:19.931 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619317+0000", 2024-04-25T20:10:19.931 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967296 2024-04-25T20:10:19.931 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.931 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.931 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:19.931 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619364+0000", 2024-04-25T20:10:19.931 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.6572000000000001e-05 2024-04-25T20:10:19.931 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.931 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.931 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:19.932 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619371+0000", 2024-04-25T20:10:19.932 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3860000000000001e-06 2024-04-25T20:10:19.932 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.932 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.932 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:19.932 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619382+0000", 2024-04-25T20:10:19.932 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0173999999999999e-05 2024-04-25T20:10:19.932 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.932 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.932 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:19.932 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624216+0000", 2024-04-25T20:10:19.932 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048340350000000004 2024-04-25T20:10:19.932 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.932 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.932 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:19.933 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624260+0000", 2024-04-25T20:10:19.933 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4558999999999998e-05 2024-04-25T20:10:19.933 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.933 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.933 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:19.933 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624333+0000", 2024-04-25T20:10:19.933 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2744999999999996e-05 2024-04-25T20:10:19.933 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.933 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.933 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:19.933 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625465+0000", 2024-04-25T20:10:19.933 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001132454 2024-04-25T20:10:19.933 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.933 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.934 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:19.934 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628365+0000", 2024-04-25T20:10:19.934 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0028994950000000002 2024-04-25T20:10:19.934 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.934 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.934 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:19.934 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628386+0000", 2024-04-25T20:10:19.934 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.1367000000000001e-05 2024-04-25T20:10:19.934 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.934 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.934 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:19.934 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628425+0000", 2024-04-25T20:10:19.934 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.9087e-05 2024-04-25T20:10:19.934 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.935 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:19.935 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.935 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.935 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.935 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:159 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:19.935 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:19.935 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 61.206296965, 2024-04-25T20:10:19.935 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090948999999999995, 2024-04-25T20:10:19.935 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:19.935 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:19.935 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:19.935 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:19.935 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:19.935 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 159 2024-04-25T20:10:19.935 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.935 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:19.936 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.936 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:19.936 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:19.936 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.936 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.936 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.936 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:19.936 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:19.936 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:19.936 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.936 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.936 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:19.936 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:19.936 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:19.937 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.937 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.937 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:19.937 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619389+0000", 2024-04-25T20:10:19.937 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.734e-06 2024-04-25T20:10:19.937 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.937 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.937 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:19.937 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619392+0000", 2024-04-25T20:10:19.937 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6219999999999998e-06 2024-04-25T20:10:19.937 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.937 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.937 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:19.937 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619403+0000", 2024-04-25T20:10:19.937 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0987e-05 2024-04-25T20:10:19.938 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.938 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.938 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:19.938 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624667+0000", 2024-04-25T20:10:19.938 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0052633970000000004 2024-04-25T20:10:19.938 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.938 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.938 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:19.938 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624741+0000", 2024-04-25T20:10:19.938 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.4583000000000001e-05 2024-04-25T20:10:19.938 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.938 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.938 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:19.938 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624857+0000", 2024-04-25T20:10:19.939 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000116118 2024-04-25T20:10:19.939 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.939 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.939 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:19.939 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.626013+0000", 2024-04-25T20:10:19.939 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001155473 2024-04-25T20:10:19.939 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.939 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.939 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:19.939 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628432+0000", 2024-04-25T20:10:19.939 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0024189379999999998 2024-04-25T20:10:19.939 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.939 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.939 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:19.940 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628454+0000", 2024-04-25T20:10:19.940 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2059999999999999e-05 2024-04-25T20:10:19.940 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:19.940 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:19.940 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:19.940 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628481+0000", 2024-04-25T20:10:19.940 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.7300999999999998e-05 2024-04-25T20:10:19.940 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.940 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:19.940 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.940 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:19.940 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:19.940 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:19.942 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:10:19.942 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-04-25T20:10:19.943 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:19.942+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:19.943 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T20:10:19.944 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-04-25T20:10:19.612084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":173004,"kb_used_data":2688,"kb_used_omap":9,"kb_used_meta":170230,"kb_avail":566058036,"statfs":{"total":579820584960,"available":579643428864,"internally_reserved":0,"allocated":2752512,"data_stored":2221762,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9555,"internal_metadata":174316205},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000664"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":40,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T20:10:12.946833+0000","last_change":"2024-04-25T20:09:41.160555+0000","last_active":"2024-04-25T20:10:12.946833+0000","last_peered":"2024-04-25T20:10:12.946833+0000","last_clean":"2024-04-25T20:10:12.946833+0000","last_became_active":"2024-04-25T20:09:41.160309+0000","last_became_peered":"2024-04-25T20:09:41.160309+0000","last_unstale":"2024-04-25T20:10:12.946833+0000","last_undegraded":"2024-04-25T20:10:12.946833+0000","last_fullsized":"2024-04-25T20:10:12.946833+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:21:47.019303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T20:10:12.946880+0000","last_change":"2024-04-25T20:09:16.141209+0000","last_active":"2024-04-25T20:10:12.946880+0000","last_peered":"2024-04-25T20:10:12.946880+0000","last_clean":"2024-04-25T20:10:12.946880+0000","last_became_active":"2024-04-25T20:09:16.140616+0000","last_became_peered":"2024-04-25T20:09:16.140616+0000","last_unstale":"2024-04-25T20:10:12.946880+0000","last_undegraded":"2024-04-25T20:10:12.946880+0000","last_fullsized":"2024-04-25T20:10:12.946880+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:31.562175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":39,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T20:10:09.423240+0000","last_change":"2024-04-25T20:10:03.784634+0000","last_active":"2024-04-25T20:10:09.423240+0000","last_peered":"2024-04-25T20:10:09.423240+0000","last_clean":"2024-04-25T20:10:09.423240+0000","last_became_active":"2024-04-25T20:10:03.784247+0000","last_became_peered":"2024-04-25T20:10:03.784247+0000","last_unstale":"2024-04-25T20:10:09.423240+0000","last_undegraded":"2024-04-25T20:10:09.423240+0000","last_fullsized":"2024-04-25T20:10:09.423240+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:35.219740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":24,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T20:10:12.400285+0000","last_change":"2024-04-25T20:09:41.938573+0000","last_active":"2024-04-25T20:10:12.400285+0000","last_peered":"2024-04-25T20:10:12.400285+0000","last_clean":"2024-04-25T20:10:12.400285+0000","last_became_active":"2024-04-25T20:09:41.938314+0000","last_became_peered":"2024-04-25T20:09:41.938314+0000","last_unstale":"2024-04-25T20:10:12.400285+0000","last_undegraded":"2024-04-25T20:10:12.400285+0000","last_fullsized":"2024-04-25T20:10:12.400285+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:01:12.669404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":182,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T20:10:12.400290+0000","last_change":"2024-04-25T20:09:17.121148+0000","last_active":"2024-04-25T20:10:12.400290+0000","last_peered":"2024-04-25T20:10:12.400290+0000","last_clean":"2024-04-25T20:10:12.400290+0000","last_became_active":"2024-04-25T20:09:17.120312+0000","last_became_peered":"2024-04-25T20:09:17.120312+0000","last_unstale":"2024-04-25T20:10:12.400290+0000","last_undegraded":"2024-04-25T20:10:12.400290+0000","last_fullsized":"2024-04-25T20:10:12.400290+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_clean_scrub_stamp":"2024-04-25T20:09:15.745353+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:48.383841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":34,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T20:10:12.923108+0000","last_change":"2024-04-25T20:09:15.772839+0000","last_active":"2024-04-25T20:10:12.923108+0000","last_peered":"2024-04-25T20:10:12.923108+0000","last_clean":"2024-04-25T20:10:12.923108+0000","last_became_active":"2024-04-25T20:09:15.772551+0000","last_became_peered":"2024-04-25T20:09:15.772551+0000","last_unstale":"2024-04-25T20:10:12.923108+0000","last_undegraded":"2024-04-25T20:10:12.923108+0000","last_fullsized":"2024-04-25T20:10:12.923108+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:55.101025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":33,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T20:10:12.400311+0000","last_change":"2024-04-25T20:09:16.141037+0000","last_active":"2024-04-25T20:10:12.400311+0000","last_peered":"2024-04-25T20:10:12.400311+0000","last_clean":"2024-04-25T20:10:12.400311+0000","last_became_active":"2024-04-25T20:09:16.140704+0000","last_became_peered":"2024-04-25T20:09:16.140704+0000","last_unstale":"2024-04-25T20:10:12.400311+0000","last_undegraded":"2024-04-25T20:10:12.400311+0000","last_fullsized":"2024-04-25T20:10:12.400311+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:22.537137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":29,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T20:10:12.947213+0000","last_change":"2024-04-25T20:09:35.912960+0000","last_active":"2024-04-25T20:10:12.947213+0000","last_peered":"2024-04-25T20:10:12.947213+0000","last_clean":"2024-04-25T20:10:12.947213+0000","last_became_active":"2024-04-25T20:09:35.912678+0000","last_became_peered":"2024-04-25T20:09:35.912678+0000","last_unstale":"2024-04-25T20:10:12.947213+0000","last_undegraded":"2024-04-25T20:10:12.947213+0000","last_fullsized":"2024-04-25T20:10:12.947213+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:28.437601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":33,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T20:10:12.400295+0000","last_change":"2024-04-25T20:09:16.141064+0000","last_active":"2024-04-25T20:10:12.400295+0000","last_peered":"2024-04-25T20:10:12.400295+0000","last_clean":"2024-04-25T20:10:12.400295+0000","last_became_active":"2024-04-25T20:09:16.140731+0000","last_became_peered":"2024-04-25T20:09:16.140731+0000","last_unstale":"2024-04-25T20:10:12.400295+0000","last_undegraded":"2024-04-25T20:10:12.400295+0000","last_fullsized":"2024-04-25T20:10:12.400295+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:18.458296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":495616,"data_stored":763120,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.622}]},{"osd":1,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.58799999999999997}]},{"osd":2,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.48099999999999998}]},{"osd":3,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.64600000000000002}]},{"osd":4,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.39700000000000002}]},{"osd":5,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.499},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.56000000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":438272,"data_stored":172752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]}]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":438272,"data_stored":172752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":438272,"data_stored":172752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]}]},{"osd":3,"up_from":0,"seq":55834574864,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":0,"seq":55834574863,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":507904,"data_stored":768570,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31540,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340300,"statfs":{"total":96636764160,"available":96604467200,"internally_reserved":0,"allocated":434176,"data_stored":171816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:10:19.945 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T20:10:20.042 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:20.042+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:20.140 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:20.140 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:20.140 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:20.140 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:20.143 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:20.141+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:20.151 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T20:10:20.244 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:20.244+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:20.344 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:20.343+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:20.359 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:20.359 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:20.359 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:20.359 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:20.359 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:20.371 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T20:10:20.446 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:20.446+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:20.483 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-25T20:10:20.523 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:10:20.523 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-04-25T20:10:20.547 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:20.545+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:20.548 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-04-25T20:10:19.612084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":173004,"kb_used_data":2688,"kb_used_omap":9,"kb_used_meta":170230,"kb_avail":566058036,"statfs":{"total":579820584960,"available":579643428864,"internally_reserved":0,"allocated":2752512,"data_stored":2221762,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9555,"internal_metadata":174316205},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000664"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":40,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T20:10:12.946833+0000","last_change":"2024-04-25T20:09:41.160555+0000","last_active":"2024-04-25T20:10:12.946833+0000","last_peered":"2024-04-25T20:10:12.946833+0000","last_clean":"2024-04-25T20:10:12.946833+0000","last_became_active":"2024-04-25T20:09:41.160309+0000","last_became_peered":"2024-04-25T20:09:41.160309+0000","last_unstale":"2024-04-25T20:10:12.946833+0000","last_undegraded":"2024-04-25T20:10:12.946833+0000","last_fullsized":"2024-04-25T20:10:12.946833+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:21:47.019303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T20:10:12.946880+0000","last_change":"2024-04-25T20:09:16.141209+0000","last_active":"2024-04-25T20:10:12.946880+0000","last_peered":"2024-04-25T20:10:12.946880+0000","last_clean":"2024-04-25T20:10:12.946880+0000","last_became_active":"2024-04-25T20:09:16.140616+0000","last_became_peered":"2024-04-25T20:09:16.140616+0000","last_unstale":"2024-04-25T20:10:12.946880+0000","last_undegraded":"2024-04-25T20:10:12.946880+0000","last_fullsized":"2024-04-25T20:10:12.946880+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:31.562175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":39,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T20:10:09.423240+0000","last_change":"2024-04-25T20:10:03.784634+0000","last_active":"2024-04-25T20:10:09.423240+0000","last_peered":"2024-04-25T20:10:09.423240+0000","last_clean":"2024-04-25T20:10:09.423240+0000","last_became_active":"2024-04-25T20:10:03.784247+0000","last_became_peered":"2024-04-25T20:10:03.784247+0000","last_unstale":"2024-04-25T20:10:09.423240+0000","last_undegraded":"2024-04-25T20:10:09.423240+0000","last_fullsized":"2024-04-25T20:10:09.423240+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:35.219740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":24,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T20:10:12.400285+0000","last_change":"2024-04-25T20:09:41.938573+0000","last_active":"2024-04-25T20:10:12.400285+0000","last_peered":"2024-04-25T20:10:12.400285+0000","last_clean":"2024-04-25T20:10:12.400285+0000","last_became_active":"2024-04-25T20:09:41.938314+0000","last_became_peered":"2024-04-25T20:09:41.938314+0000","last_unstale":"2024-04-25T20:10:12.400285+0000","last_undegraded":"2024-04-25T20:10:12.400285+0000","last_fullsized":"2024-04-25T20:10:12.400285+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:01:12.669404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":182,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T20:10:12.400290+0000","last_change":"2024-04-25T20:09:17.121148+0000","last_active":"2024-04-25T20:10:12.400290+0000","last_peered":"2024-04-25T20:10:12.400290+0000","last_clean":"2024-04-25T20:10:12.400290+0000","last_became_active":"2024-04-25T20:09:17.120312+0000","last_became_peered":"2024-04-25T20:09:17.120312+0000","last_unstale":"2024-04-25T20:10:12.400290+0000","last_undegraded":"2024-04-25T20:10:12.400290+0000","last_fullsized":"2024-04-25T20:10:12.400290+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_clean_scrub_stamp":"2024-04-25T20:09:15.745353+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:48.383841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":34,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T20:10:12.923108+0000","last_change":"2024-04-25T20:09:15.772839+0000","last_active":"2024-04-25T20:10:12.923108+0000","last_peered":"2024-04-25T20:10:12.923108+0000","last_clean":"2024-04-25T20:10:12.923108+0000","last_became_active":"2024-04-25T20:09:15.772551+0000","last_became_peered":"2024-04-25T20:09:15.772551+0000","last_unstale":"2024-04-25T20:10:12.923108+0000","last_undegraded":"2024-04-25T20:10:12.923108+0000","last_fullsized":"2024-04-25T20:10:12.923108+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:55.101025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":33,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T20:10:12.400311+0000","last_change":"2024-04-25T20:09:16.141037+0000","last_active":"2024-04-25T20:10:12.400311+0000","last_peered":"2024-04-25T20:10:12.400311+0000","last_clean":"2024-04-25T20:10:12.400311+0000","last_became_active":"2024-04-25T20:09:16.140704+0000","last_became_peered":"2024-04-25T20:09:16.140704+0000","last_unstale":"2024-04-25T20:10:12.400311+0000","last_undegraded":"2024-04-25T20:10:12.400311+0000","last_fullsized":"2024-04-25T20:10:12.400311+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:22.537137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":29,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T20:10:12.947213+0000","last_change":"2024-04-25T20:09:35.912960+0000","last_active":"2024-04-25T20:10:12.947213+0000","last_peered":"2024-04-25T20:10:12.947213+0000","last_clean":"2024-04-25T20:10:12.947213+0000","last_became_active":"2024-04-25T20:09:35.912678+0000","last_became_peered":"2024-04-25T20:09:35.912678+0000","last_unstale":"2024-04-25T20:10:12.947213+0000","last_undegraded":"2024-04-25T20:10:12.947213+0000","last_fullsized":"2024-04-25T20:10:12.947213+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:28.437601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":33,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T20:10:12.400295+0000","last_change":"2024-04-25T20:09:16.141064+0000","last_active":"2024-04-25T20:10:12.400295+0000","last_peered":"2024-04-25T20:10:12.400295+0000","last_clean":"2024-04-25T20:10:12.400295+0000","last_became_active":"2024-04-25T20:09:16.140731+0000","last_became_peered":"2024-04-25T20:09:16.140731+0000","last_unstale":"2024-04-25T20:10:12.400295+0000","last_undegraded":"2024-04-25T20:10:12.400295+0000","last_fullsized":"2024-04-25T20:10:12.400295+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:18.458296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":495616,"data_stored":763120,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.622}]},{"osd":1,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.58799999999999997}]},{"osd":2,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.48099999999999998}]},{"osd":3,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.64600000000000002}]},{"osd":4,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.39700000000000002}]},{"osd":5,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.499},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.56000000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":438272,"data_stored":172752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]}]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":438272,"data_stored":172752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":438272,"data_stored":172752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]}]},{"osd":3,"up_from":0,"seq":55834574864,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":0,"seq":55834574863,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":507904,"data_stored":768570,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31540,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340300,"statfs":{"total":96636764160,"available":96604467200,"internally_reserved":0,"allocated":434176,"data_stored":171816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:10:20.549 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-25T20:10:20.549 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T20:10:20.549 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-04-25T20:10:20.549 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-04-25T20:10:20.550 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-04-25T20:10:20.550 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-04-25T20:10:20.584 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:20.584 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:20.584 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:20.584 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:10:20.585 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:20.598 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T20:10:20.682 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:20.681+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:20.749 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:20.749+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:20.756 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:20.757 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-04-25T20:10:20.757 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:20.813 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:20.813 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:20.813 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:20.813 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:20.825 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T20:10:20.850 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:20.850+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:20.951 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:20.951+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:20.994 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:20.994 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:20.994 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:20.994 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:20.994 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:21.007 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T20:10:21.051 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:21.051+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:21.153 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:21.153+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:21.177 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:21.177 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:21.177 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:21.177 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-04-25T20:10:21.178 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.178 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:30 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'16, mlcod=16'14)", 2024-04-25T20:10:21.178 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:10:21.178 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 64.008341307999999, 2024-04-25T20:10:21.178 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0054087670000000001, 2024-04-25T20:10:21.178 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:21.178 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:21.178 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:21.178 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.178 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:21.178 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:10:21.178 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.178 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.178 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.179 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:21.179 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:10:21.179 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.179 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.179 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.179 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:21.179 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167142+0000", 2024-04-25T20:10:21.179 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999976 2024-04-25T20:10:21.179 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.179 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.179 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:21.179 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167162+0000", 2024-04-25T20:10:21.179 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0109e-05 2024-04-25T20:10:21.179 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.179 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.179 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:21.180 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167188+0000", 2024-04-25T20:10:21.180 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6146e-05 2024-04-25T20:10:21.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.180 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.180 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:21.180 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167199+0000", 2024-04-25T20:10:21.180 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0767e-05 2024-04-25T20:10:21.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.180 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.180 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:21.180 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167256+0000", 2024-04-25T20:10:21.180 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7392000000000002e-05 2024-04-25T20:10:21.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.180 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.180 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:21.181 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167283+0000", 2024-04-25T20:10:21.181 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6577e-05 2024-04-25T20:10:21.181 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.181 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.181 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:21.181 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.172510+0000", 2024-04-25T20:10:21.181 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0052276759999999997 2024-04-25T20:10:21.181 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.181 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.181 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:21.181 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.172553+0000", 2024-04-25T20:10:21.181 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2444999999999998e-05 2024-04-25T20:10:21.181 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.181 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:21.181 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.182 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.182 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.182 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:31 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-04-25T20:10:21.182 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:21.182 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 64.007725230000005, 2024-04-25T20:10:21.182 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0055055479999999999, 2024-04-25T20:10:21.182 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:21.182 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:21.182 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:21.182 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.182 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:21.182 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:21.182 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.182 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.182 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.182 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:21.182 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:21.183 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.183 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.183 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.183 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:21.183 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:21.183 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:21.183 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.183 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.183 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:21.183 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167766+0000", 2024-04-25T20:10:21.183 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.122e-06 2024-04-25T20:10:21.183 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.183 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.183 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:21.184 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167776+0000", 2024-04-25T20:10:21.184 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0515e-05 2024-04-25T20:10:21.184 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.184 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.184 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:21.184 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167782+0000", 2024-04-25T20:10:21.184 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1149999999999999e-06 2024-04-25T20:10:21.184 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.184 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.184 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:21.184 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167837+0000", 2024-04-25T20:10:21.184 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4763999999999999e-05 2024-04-25T20:10:21.184 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.184 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.184 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:21.184 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167855+0000", 2024-04-25T20:10:21.184 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7589e-05 2024-04-25T20:10:21.185 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.185 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.185 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:21.185 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173221+0000", 2024-04-25T20:10:21.185 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.005365903 2024-04-25T20:10:21.185 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.185 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.185 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:21.185 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173266+0000", 2024-04-25T20:10:21.185 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5059999999999999e-05 2024-04-25T20:10:21.185 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.185 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:21.185 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.185 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.185 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.185 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:32 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-04-25T20:10:21.186 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:21.186 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 64.007350044999995, 2024-04-25T20:10:21.186 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0051558239999999998, 2024-04-25T20:10:21.186 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:21.186 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:21.186 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:21.186 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.186 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:21.186 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:21.186 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.186 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.186 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.186 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:21.186 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:21.187 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.187 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:21.187 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:21.187 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:21.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.187 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:21.187 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168173+0000", 2024-04-25T20:10:21.187 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7663999999999997e-05 2024-04-25T20:10:21.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.187 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:21.187 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168182+0000", 2024-04-25T20:10:21.187 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0650000000000005e-06 2024-04-25T20:10:21.188 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.188 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.188 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:21.188 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168188+0000", 2024-04-25T20:10:21.188 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0589999999999999e-06 2024-04-25T20:10:21.188 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.188 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.188 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:21.188 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168239+0000", 2024-04-25T20:10:21.188 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1467e-05 2024-04-25T20:10:21.188 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.188 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.188 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:21.189 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168260+0000", 2024-04-25T20:10:21.189 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0531e-05 2024-04-25T20:10:21.189 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.189 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.189 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:21.189 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173268+0000", 2024-04-25T20:10:21.189 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0050087980000000001 2024-04-25T20:10:21.189 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.189 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.189 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:21.189 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173291+0000", 2024-04-25T20:10:21.189 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.268e-05 2024-04-25T20:10:21.189 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.189 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:21.189 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.190 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.190 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.190 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:33 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'19, mlcod=16'14)", 2024-04-25T20:10:21.190 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:10:21.190 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 64.006839626000001, 2024-04-25T20:10:21.190 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046697049999999997, 2024-04-25T20:10:21.190 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:21.190 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:21.190 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:21.190 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.190 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:21.190 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:10:21.190 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.191 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.191 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.191 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:21.191 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:10:21.191 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.191 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.191 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.191 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:21.191 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168645+0000", 2024-04-25T20:10:21.191 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:21.191 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.191 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.191 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:21.192 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168651+0000", 2024-04-25T20:10:21.192 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1079999999999998e-06 2024-04-25T20:10:21.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.192 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.192 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:21.192 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168660+0000", 2024-04-25T20:10:21.192 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0010000000000007e-06 2024-04-25T20:10:21.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.192 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.192 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:21.192 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168666+0000", 2024-04-25T20:10:21.192 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7780000000000004e-06 2024-04-25T20:10:21.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.192 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.192 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:21.193 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168701+0000", 2024-04-25T20:10:21.193 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4490999999999999e-05 2024-04-25T20:10:21.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.193 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.193 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:21.193 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168729+0000", 2024-04-25T20:10:21.193 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8476999999999999e-05 2024-04-25T20:10:21.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.193 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.193 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:21.193 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173294+0000", 2024-04-25T20:10:21.193 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0045650730000000002 2024-04-25T20:10:21.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.193 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.193 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:21.193 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173315+0000", 2024-04-25T20:10:21.193 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1245000000000001e-05 2024-04-25T20:10:21.194 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.194 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:21.194 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.194 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.194 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.194 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:34 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'20, mlcod=16'14)", 2024-04-25T20:10:21.194 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:10:21.194 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 64.006340696999999, 2024-04-25T20:10:21.194 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041943309999999999, 2024-04-25T20:10:21.194 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:21.194 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:21.194 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:21.194 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.194 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:21.194 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:10:21.195 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.195 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.195 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.195 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:21.195 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:10:21.195 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.195 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.195 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.195 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:21.195 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169144+0000", 2024-04-25T20:10:21.195 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:21.195 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.195 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.195 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:21.195 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169150+0000", 2024-04-25T20:10:21.195 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9150000000000001e-06 2024-04-25T20:10:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169159+0000", 2024-04-25T20:10:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8899999999999996e-06 2024-04-25T20:10:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169169+0000", 2024-04-25T20:10:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0441e-05 2024-04-25T20:10:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169206+0000", 2024-04-25T20:10:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6816000000000001e-05 2024-04-25T20:10:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.196 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169234+0000", 2024-04-25T20:10:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8008999999999999e-05 2024-04-25T20:10:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173317+0000", 2024-04-25T20:10:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0040830160000000001 2024-04-25T20:10:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173339+0000", 2024-04-25T20:10:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1897000000000001e-05 2024-04-25T20:10:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.197 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:21.198 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.198 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.198 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:35 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'14)", 2024-04-25T20:10:21.198 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:10:21.198 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 64.005848377999996, 2024-04-25T20:10:21.198 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037236529999999999, 2024-04-25T20:10:21.198 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:21.198 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:21.198 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:21.200 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.200 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:21.200 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:10:21.200 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.200 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.200 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:21.200 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:10:21.200 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.201 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.201 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:21.201 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169636+0000", 2024-04-25T20:10:21.201 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:21.201 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.201 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.201 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:21.201 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169678+0000", 2024-04-25T20:10:21.201 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1752000000000003e-05 2024-04-25T20:10:21.201 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.201 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.201 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:21.201 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169688+0000", 2024-04-25T20:10:21.201 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0098000000000001e-05 2024-04-25T20:10:21.201 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.202 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.202 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:21.202 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169694+0000", 2024-04-25T20:10:21.202 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2349999999999996e-06 2024-04-25T20:10:21.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.202 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.202 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:21.202 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169733+0000", 2024-04-25T20:10:21.202 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8481e-05 2024-04-25T20:10:21.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.202 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.202 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:21.202 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169750+0000", 2024-04-25T20:10:21.202 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7509000000000001e-05 2024-04-25T20:10:21.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.203 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.203 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:21.203 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173340+0000", 2024-04-25T20:10:21.203 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0035898449999999999 2024-04-25T20:10:21.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.203 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.203 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:21.203 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173361+0000", 2024-04-25T20:10:21.203 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.039e-05 2024-04-25T20:10:21.203 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.203 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:21.203 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.204 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.204 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:36 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'22, mlcod=16'14)", 2024-04-25T20:10:21.204 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:21.204 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 64.005348318000003, 2024-04-25T20:10:21.204 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032472120000000002, 2024-04-25T20:10:21.204 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:21.204 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:21.204 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:21.204 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.204 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:21.204 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:21.204 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.204 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.205 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.205 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:21.205 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:21.205 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.205 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.205 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:21.205 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:21.205 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:21.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.205 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.205 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:21.205 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170152+0000", 2024-04-25T20:10:21.205 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5798e-05 2024-04-25T20:10:21.206 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.206 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.206 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:21.206 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170161+0000", 2024-04-25T20:10:21.206 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.064e-06 2024-04-25T20:10:21.206 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.206 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.206 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:21.206 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170167+0000", 2024-04-25T20:10:21.206 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8390000000000003e-06 2024-04-25T20:10:21.206 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.206 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.206 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:21.207 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170202+0000", 2024-04-25T20:10:21.207 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.5120999999999998e-05 2024-04-25T20:10:21.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.207 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.207 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:21.207 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170219+0000", 2024-04-25T20:10:21.207 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6929999999999999e-05 2024-04-25T20:10:21.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.207 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.207 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:21.207 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173362+0000", 2024-04-25T20:10:21.207 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0031427619999999999 2024-04-25T20:10:21.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.208 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.208 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:21.208 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173384+0000", 2024-04-25T20:10:21.208 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2237000000000001e-05 2024-04-25T20:10:21.208 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.208 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:21.208 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.208 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.208 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.208 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:40 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'26, mlcod=16'24)", 2024-04-25T20:10:21.208 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:10:21.208 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 63.998310406000002, 2024-04-25T20:10:21.208 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028727330000000001, 2024-04-25T20:10:21.208 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:21.208 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:21.209 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:21.209 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.209 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:21.209 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:10:21.209 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.209 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.209 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.209 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:21.209 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:10:21.209 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.209 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.209 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.209 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:21.209 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177173+0000", 2024-04-25T20:10:21.209 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999976 2024-04-25T20:10:21.210 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.210 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.210 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:21.210 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177186+0000", 2024-04-25T20:10:21.210 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3533000000000001e-05 2024-04-25T20:10:21.210 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.210 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.210 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:21.210 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177209+0000", 2024-04-25T20:10:21.210 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3280000000000001e-05 2024-04-25T20:10:21.210 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.210 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.210 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:21.211 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177220+0000", 2024-04-25T20:10:21.211 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0465000000000001e-05 2024-04-25T20:10:21.211 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.211 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.211 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:21.211 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177264+0000", 2024-04-25T20:10:21.211 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4413999999999999e-05 2024-04-25T20:10:21.211 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.211 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.211 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:21.211 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177289+0000", 2024-04-25T20:10:21.211 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4783999999999999e-05 2024-04-25T20:10:21.211 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.211 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.212 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:21.212 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.179990+0000", 2024-04-25T20:10:21.212 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027012939999999999 2024-04-25T20:10:21.212 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.212 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.212 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:21.212 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.180048+0000", 2024-04-25T20:10:21.212 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7299000000000001e-05 2024-04-25T20:10:21.212 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.212 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:21.212 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.212 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.212 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.212 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:67 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'41, mlcod=16'39)", 2024-04-25T20:10:21.212 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:10:21.213 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 63.963548914, 2024-04-25T20:10:21.213 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064192549999999996, 2024-04-25T20:10:21.213 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:21.213 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:21.213 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:21.213 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.213 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:21.213 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:10:21.213 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.213 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.213 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.213 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:21.213 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:10:21.213 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.213 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.214 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.214 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:21.214 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211935+0000", 2024-04-25T20:10:21.214 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999981 2024-04-25T20:10:21.214 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.214 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.214 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:21.214 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211948+0000", 2024-04-25T20:10:21.214 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3531e-05 2024-04-25T20:10:21.214 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.214 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.214 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:21.214 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211967+0000", 2024-04-25T20:10:21.214 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8326999999999999e-05 2024-04-25T20:10:21.214 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.215 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.215 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:21.215 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211975+0000", 2024-04-25T20:10:21.215 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8589999999999994e-06 2024-04-25T20:10:21.215 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.215 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.215 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:21.215 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212020+0000", 2024-04-25T20:10:21.215 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4626999999999998e-05 2024-04-25T20:10:21.216 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.216 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.216 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:21.216 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212045+0000", 2024-04-25T20:10:21.216 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4508999999999999e-05 2024-04-25T20:10:21.216 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.216 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.216 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:21.217 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218316+0000", 2024-04-25T20:10:21.217 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0062711640000000001 2024-04-25T20:10:21.217 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.217 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.217 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:21.217 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218356+0000", 2024-04-25T20:10:21.217 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9993000000000001e-05 2024-04-25T20:10:21.217 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.217 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:21.217 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.217 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.217 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.218 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:68 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-04-25T20:10:21.218 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:21.218 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 63.963099900000003, 2024-04-25T20:10:21.218 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011231365, 2024-04-25T20:10:21.218 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:21.218 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:21.219 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:21.219 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.219 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:21.219 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:21.219 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.219 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.219 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.219 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:21.219 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:21.219 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.219 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.219 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.219 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:21.219 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:21.219 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:21.219 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.220 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.220 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:21.220 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212392+0000", 2024-04-25T20:10:21.220 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.79e-06 2024-04-25T20:10:21.220 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.220 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.220 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:21.220 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212406+0000", 2024-04-25T20:10:21.220 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3577000000000001e-05 2024-04-25T20:10:21.220 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.220 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.220 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:21.220 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212414+0000", 2024-04-25T20:10:21.220 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6510000000000001e-06 2024-04-25T20:10:21.221 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.221 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.221 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:21.221 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217114+0000", 2024-04-25T20:10:21.221 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047005379999999998 2024-04-25T20:10:21.221 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.221 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.221 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:21.221 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217141+0000", 2024-04-25T20:10:21.221 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6984000000000001e-05 2024-04-25T20:10:21.221 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.221 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.221 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:21.221 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223563+0000", 2024-04-25T20:10:21.221 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064216999999999998 2024-04-25T20:10:21.222 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.222 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.222 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:21.222 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223617+0000", 2024-04-25T20:10:21.222 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4011000000000002e-05 2024-04-25T20:10:21.222 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.222 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:21.222 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.222 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.222 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.222 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:69 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-04-25T20:10:21.222 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:10:21.222 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 63.962567710000002, 2024-04-25T20:10:21.222 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.010730094000000001, 2024-04-25T20:10:21.223 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:21.223 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:21.223 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:21.223 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.223 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:21.223 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:10:21.223 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.223 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.223 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.223 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:21.223 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:10:21.223 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.223 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.223 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.223 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:21.223 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212917+0000", 2024-04-25T20:10:21.224 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:21.224 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.224 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.224 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:21.224 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212957+0000", 2024-04-25T20:10:21.224 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0692000000000003e-05 2024-04-25T20:10:21.224 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.224 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.224 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:21.224 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212973+0000", 2024-04-25T20:10:21.224 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.575e-05 2024-04-25T20:10:21.224 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.224 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.224 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:21.224 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212980+0000", 2024-04-25T20:10:21.225 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1679999999999998e-06 2024-04-25T20:10:21.225 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.225 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.225 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:21.225 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217449+0000", 2024-04-25T20:10:21.225 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.004468367 2024-04-25T20:10:21.225 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.225 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.225 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:21.225 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217470+0000", 2024-04-25T20:10:21.225 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0749000000000001e-05 2024-04-25T20:10:21.225 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.225 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.226 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:21.226 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223621+0000", 2024-04-25T20:10:21.226 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0061512880000000004 2024-04-25T20:10:21.226 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.226 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.226 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:21.226 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223648+0000", 2024-04-25T20:10:21.226 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6961000000000002e-05 2024-04-25T20:10:21.227 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.227 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:21.227 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.227 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.227 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.227 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:70 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'44, mlcod=16'39)", 2024-04-25T20:10:21.227 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:21.227 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 63.961808982000001, 2024-04-25T20:10:21.227 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.013553358, 2024-04-25T20:10:21.227 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:21.227 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:21.227 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:21.227 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.227 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:21.228 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:21.228 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.228 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.228 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.228 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:21.228 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:21.228 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.228 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.228 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.228 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:21.228 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:21.228 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:21.228 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.228 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.228 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:21.229 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213682+0000", 2024-04-25T20:10:21.229 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3439999999999997e-06 2024-04-25T20:10:21.229 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.229 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.229 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:21.229 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213693+0000", 2024-04-25T20:10:21.229 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1049999999999999e-05 2024-04-25T20:10:21.229 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.229 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.229 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:21.229 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213699+0000", 2024-04-25T20:10:21.229 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9399999999999999e-06 2024-04-25T20:10:21.229 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.229 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.229 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:21.230 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218086+0000", 2024-04-25T20:10:21.230 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0043871240000000001 2024-04-25T20:10:21.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.230 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.230 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:21.231 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218106+0000", 2024-04-25T20:10:21.231 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.997e-05 2024-04-25T20:10:21.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.231 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.231 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:21.232 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.227186+0000", 2024-04-25T20:10:21.232 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.009079574 2024-04-25T20:10:21.232 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.232 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.232 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:21.232 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.227230+0000", 2024-04-25T20:10:21.232 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4147999999999998e-05 2024-04-25T20:10:21.232 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.232 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:21.232 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.232 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.232 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.232 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:106 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-04-25T20:10:21.232 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:10:21.232 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 63.590669882, 2024-04-25T20:10:21.233 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002936756, 2024-04-25T20:10:21.233 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:21.233 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:21.233 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:21.233 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.233 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:21.233 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:10:21.233 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.233 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.233 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.233 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:21.233 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:10:21.233 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.233 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.233 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.234 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:21.234 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584815+0000", 2024-04-25T20:10:21.234 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:21.234 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.234 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.234 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:21.234 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584860+0000", 2024-04-25T20:10:21.234 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5161999999999999e-05 2024-04-25T20:10:21.234 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.234 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.234 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:21.234 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584874+0000", 2024-04-25T20:10:21.234 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4464000000000001e-05 2024-04-25T20:10:21.235 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.235 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.235 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:21.235 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584881+0000", 2024-04-25T20:10:21.235 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.409e-06 2024-04-25T20:10:21.235 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.235 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.235 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:21.235 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584915+0000", 2024-04-25T20:10:21.235 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4007000000000002e-05 2024-04-25T20:10:21.235 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.235 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.236 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:21.236 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584933+0000", 2024-04-25T20:10:21.236 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7737999999999999e-05 2024-04-25T20:10:21.236 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.236 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.236 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:21.236 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587717+0000", 2024-04-25T20:10:21.236 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002784711 2024-04-25T20:10:21.236 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.236 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.236 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:21.236 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587752+0000", 2024-04-25T20:10:21.236 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4915e-05 2024-04-25T20:10:21.236 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.236 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:21.237 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.237 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.237 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.237 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:107 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'69, mlcod=16'64)", 2024-04-25T20:10:21.237 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:10:21.237 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 63.590594580000001, 2024-04-25T20:10:21.237 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028906270000000002, 2024-04-25T20:10:21.237 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:21.237 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:21.237 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:21.237 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.237 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:21.237 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:10:21.237 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.238 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.238 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.238 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:21.238 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:10:21.238 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.238 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.238 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.238 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:21.238 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584890+0000", 2024-04-25T20:10:21.238 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:21.238 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.238 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.238 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:21.238 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584895+0000", 2024-04-25T20:10:21.239 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2069999999999997e-06 2024-04-25T20:10:21.239 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.239 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.239 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:21.239 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584906+0000", 2024-04-25T20:10:21.239 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.117e-05 2024-04-25T20:10:21.239 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.239 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.239 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:21.239 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584910+0000", 2024-04-25T20:10:21.239 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4190000000000004e-06 2024-04-25T20:10:21.239 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.240 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.240 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:21.240 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585310+0000", 2024-04-25T20:10:21.240 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00040002700000000001 2024-04-25T20:10:21.240 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.240 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.240 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:21.240 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585329+0000", 2024-04-25T20:10:21.240 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8838000000000002e-05 2024-04-25T20:10:21.240 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.240 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.240 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:21.240 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587754+0000", 2024-04-25T20:10:21.240 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024252689999999999 2024-04-25T20:10:21.240 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.241 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.241 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:21.241 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587781+0000", 2024-04-25T20:10:21.241 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7206999999999999e-05 2024-04-25T20:10:21.241 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.241 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:21.241 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.241 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.241 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.241 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:108 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'70, mlcod=16'64)", 2024-04-25T20:10:21.241 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:10:21.241 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 63.590569428999999, 2024-04-25T20:10:21.241 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028924879999999999, 2024-04-25T20:10:21.241 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:21.241 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:21.241 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:21.242 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.242 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:21.242 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:10:21.242 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.242 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.242 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.242 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:21.242 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:10:21.242 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.242 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.242 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.242 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:21.243 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:10:21.243 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:21.243 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.243 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.243 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:21.243 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584919+0000", 2024-04-25T20:10:21.243 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.557e-06 2024-04-25T20:10:21.243 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.243 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.243 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:21.243 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584925+0000", 2024-04-25T20:10:21.243 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8619999999999999e-06 2024-04-25T20:10:21.243 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.243 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.243 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:21.244 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584931+0000", 2024-04-25T20:10:21.244 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2539999999999998e-06 2024-04-25T20:10:21.244 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.244 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.244 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:21.244 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585550+0000", 2024-04-25T20:10:21.244 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00061821800000000002 2024-04-25T20:10:21.244 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.244 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.244 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:21.244 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585567+0000", 2024-04-25T20:10:21.244 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7499e-05 2024-04-25T20:10:21.244 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.244 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.244 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:21.245 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587783+0000", 2024-04-25T20:10:21.245 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0022156200000000002 2024-04-25T20:10:21.245 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.245 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.245 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:21.245 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587808+0000", 2024-04-25T20:10:21.245 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5729e-05 2024-04-25T20:10:21.246 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.246 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:21.246 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.246 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.246 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.246 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:112 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'74, mlcod=16'66)", 2024-04-25T20:10:21.247 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:10:21.247 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 63.588411807, 2024-04-25T20:10:21.247 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00318724, 2024-04-25T20:10:21.247 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:21.247 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:21.248 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:21.248 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.248 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:21.248 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:10:21.248 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.248 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.249 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.249 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:21.249 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:10:21.249 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.249 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.249 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.250 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:21.250 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587073+0000", 2024-04-25T20:10:21.250 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:21.250 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.250 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.251 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:21.251 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587136+0000", 2024-04-25T20:10:21.251 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2928999999999993e-05 2024-04-25T20:10:21.251 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.251 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.251 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:21.252 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587147+0000", 2024-04-25T20:10:21.252 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1182999999999999e-05 2024-04-25T20:10:21.252 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.252 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.252 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:21.253 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587153+0000", 2024-04-25T20:10:21.253 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9059999999999999e-06 2024-04-25T20:10:21.253 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.253 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.253 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:21.253 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587306+0000", 2024-04-25T20:10:21.254 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000152848 2024-04-25T20:10:21.254 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.254 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.254 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:21.254 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587330+0000", 2024-04-25T20:10:21.254 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4304e-05 2024-04-25T20:10:21.255 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.255 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.255 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:21.255 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.590230+0000", 2024-04-25T20:10:21.255 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029004629999999998 2024-04-25T20:10:21.256 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.256 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.256 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:21.256 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.590261+0000", 2024-04-25T20:10:21.256 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0323000000000001e-05 2024-04-25T20:10:21.256 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.257 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:21.257 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.257 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.257 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.257 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:118 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'80, mlcod=16'78)", 2024-04-25T20:10:21.257 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:10:21.258 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 63.583625228000002, 2024-04-25T20:10:21.258 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029618959999999999, 2024-04-25T20:10:21.258 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:21.258 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:21.259 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:21.259 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.259 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:21.259 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:10:21.259 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.259 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.260 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.260 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:21.260 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:10:21.260 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.260 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.260 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.261 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:21.261 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591859+0000", 2024-04-25T20:10:21.261 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:21.261 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.261 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.261 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:21.262 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591918+0000", 2024-04-25T20:10:21.262 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8842999999999999e-05 2024-04-25T20:10:21.262 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.262 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.262 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:21.263 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591933+0000", 2024-04-25T20:10:21.263 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4600000000000001e-05 2024-04-25T20:10:21.263 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.263 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.263 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:21.263 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591939+0000", 2024-04-25T20:10:21.264 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4559999999999997e-06 2024-04-25T20:10:21.264 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.264 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.264 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:21.264 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591990+0000", 2024-04-25T20:10:21.264 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0318999999999997e-05 2024-04-25T20:10:21.264 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.265 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.265 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:21.265 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.592008+0000", 2024-04-25T20:10:21.265 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8819000000000001e-05 2024-04-25T20:10:21.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.266 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.266 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:21.266 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.594784+0000", 2024-04-25T20:10:21.266 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027750850000000001 2024-04-25T20:10:21.266 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.266 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.267 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:21.267 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.594822+0000", 2024-04-25T20:10:21.267 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8522000000000001e-05 2024-04-25T20:10:21.267 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.267 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:21.267 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.267 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.267 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.267 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:139 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'97, mlcod=17'87)", 2024-04-25T20:10:21.268 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:10:21.268 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 63.049588622999998, 2024-04-25T20:10:21.268 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048546290000000001, 2024-04-25T20:10:21.268 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:21.268 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:21.268 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:21.268 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.268 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:21.268 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:10:21.268 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.268 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.268 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.268 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:21.268 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:10:21.268 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.269 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.269 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.269 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:21.269 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125896+0000", 2024-04-25T20:10:21.269 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:21.269 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.269 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.269 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:21.269 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125908+0000", 2024-04-25T20:10:21.269 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.137e-05 2024-04-25T20:10:21.269 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.269 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.269 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:21.269 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125915+0000", 2024-04-25T20:10:21.269 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3939999999999998e-06 2024-04-25T20:10:21.269 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.270 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.270 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:21.270 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125922+0000", 2024-04-25T20:10:21.270 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1930000000000004e-06 2024-04-25T20:10:21.270 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.270 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.270 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:21.270 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125959+0000", 2024-04-25T20:10:21.270 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6701e-05 2024-04-25T20:10:21.270 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.270 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.270 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:21.271 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125972+0000", 2024-04-25T20:10:21.271 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2578000000000001e-05 2024-04-25T20:10:21.271 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.271 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.271 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:21.271 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130719+0000", 2024-04-25T20:10:21.271 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047475800000000004 2024-04-25T20:10:21.271 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.271 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.271 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:21.271 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130751+0000", 2024-04-25T20:10:21.271 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2283000000000002e-05 2024-04-25T20:10:21.271 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.271 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:21.272 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.272 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.272 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.272 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:140 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'98, mlcod=17'87)", 2024-04-25T20:10:21.272 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:21.272 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 63.049163272000001, 2024-04-25T20:10:21.272 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048310990000000002, 2024-04-25T20:10:21.272 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:21.272 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:21.272 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:21.272 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.272 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:21.272 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:21.272 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.272 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.273 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.273 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:21.273 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:21.273 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.273 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.273 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.273 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:21.273 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:21.273 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:21.273 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.273 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.273 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:21.273 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126427+0000", 2024-04-25T20:10:21.273 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000105572 2024-04-25T20:10:21.273 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.273 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.274 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:21.274 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126435+0000", 2024-04-25T20:10:21.274 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1920000000000005e-06 2024-04-25T20:10:21.274 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.274 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.274 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:21.274 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126439+0000", 2024-04-25T20:10:21.274 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6950000000000002e-06 2024-04-25T20:10:21.274 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.274 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.274 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:21.274 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126471+0000", 2024-04-25T20:10:21.274 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1699000000000003e-05 2024-04-25T20:10:21.274 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.274 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.275 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:21.275 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126484+0000", 2024-04-25T20:10:21.275 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.343e-05 2024-04-25T20:10:21.275 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.275 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.275 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:21.275 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131131+0000", 2024-04-25T20:10:21.275 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046471480000000003 2024-04-25T20:10:21.275 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.275 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.275 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:21.275 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131153+0000", 2024-04-25T20:10:21.275 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1753e-05 2024-04-25T20:10:21.275 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.275 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:21.276 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.276 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.276 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.276 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:141 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'99, mlcod=17'87)", 2024-04-25T20:10:21.276 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:21.276 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 63.048508327999997, 2024-04-25T20:10:21.276 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.004496847, 2024-04-25T20:10:21.276 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:21.276 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:21.276 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:21.276 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.276 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:21.276 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:21.276 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.277 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.277 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.277 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:21.277 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:21.277 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:21.277 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.277 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.277 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:21.277 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:21.277 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:21.277 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.277 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.277 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:21.277 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126980+0000", 2024-04-25T20:10:21.277 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.619e-06 2024-04-25T20:10:21.278 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.278 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.278 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:21.278 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126987+0000", 2024-04-25T20:10:21.278 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7220000000000002e-06 2024-04-25T20:10:21.278 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.278 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.278 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:21.278 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126992+0000", 2024-04-25T20:10:21.278 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8679999999999998e-06 2024-04-25T20:10:21.278 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.279 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.279 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:21.279 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130772+0000", 2024-04-25T20:10:21.279 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037803120000000001 2024-04-25T20:10:21.279 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.279 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.279 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:21.279 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130786+0000", 2024-04-25T20:10:21.279 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3592e-05 2024-04-25T20:10:21.279 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.279 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.279 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:21.279 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131451+0000", 2024-04-25T20:10:21.279 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000665206 2024-04-25T20:10:21.279 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:21.280 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:21.280 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:21.280 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131474+0000", 2024-04-25T20:10:21.280 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2957999999999999e-05 2024-04-25T20:10:21.280 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.280 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:21.280 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.280 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:21.280 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:21.280 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:21.281 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T20:10:21.282 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:21.282+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:21.368 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T20:10:21.382 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:21.382+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:21.483 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:21.483+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:21.492 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:21.493 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:21.493 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:21.493 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:21.504 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is unset 2024-04-25T20:10:21.506 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T20:10:21.509 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:10:21.509 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:10:21.541 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:10:21.542 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:10:21.574 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:10:21.575 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:10:21.584 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:21.583+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:21.606 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:10:21.607 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:10:21.640 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:10:21.640 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:10:21.672 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:10:21.673 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:10:21.675 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:21.675 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:21.685 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:21.685+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:21.687 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T20:10:21.704 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:10:21.704 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:10:21.736 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:10:21.736 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:10:21.786 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:21.787+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:21.817 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:21.818 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:21.827 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T20:10:21.887 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:21.887+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:21.936 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:21.936 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:21.945 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T20:10:21.988 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:21.988+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:22.089 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:22.090+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:22.111 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:22.111 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:22.123 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T20:10:22.190 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:22.191+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:22.231 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:22.231 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:22.241 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T20:10:22.292 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:22.292+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:22.354 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:22.355 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:22.364 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T20:10:22.392 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:22.392+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:22.493 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:22.492+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:22.508 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:22.508 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:22.517 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T20:10:22.594 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:22.594+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:22.618 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:22.618 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:22.627 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T20:10:22.695 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:22.696+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:22.729 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:22.729 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:22.737 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T20:10:22.797 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:22.796+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:22.879 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:22.879 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:22.887 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T20:10:22.897 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:22.897+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:22.988 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:22.989 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:22.997 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T20:10:22.998 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:22.999+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:23.099 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:23.099 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:23.099 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:23.099+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:23.107 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T20:10:23.200 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:23.201+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:23.208 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:23.208 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:23.217 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T20:10:23.301 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:23.301+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:23.358 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:23.358 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:23.367 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T20:10:23.402 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:23.402+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:23.468 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:23.468 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:23.476 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T20:10:23.504 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:23.503+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:23.524 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-25T20:10:23.576 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:23.576 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:23.584 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T20:10:23.605 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:23.604+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:23.686 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:23.686 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:23.695 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T20:10:23.705 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:23.705+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:23.752 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T20:10:23.806 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:23.806+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:23.844 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:23.844 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:23.858 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T20:10:23.906 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:23.907+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:23.951 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:10:23.951 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:10:23.984 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:10:23.985 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:10:24.008 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:24.007+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:24.017 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:10:24.017 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:10:24.029 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:24.029 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:24.029 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:24.029 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:24.029 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:24.042 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T20:10:24.049 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:10:24.050 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:10:24.081 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:10:24.082 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:10:24.108 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:24.108+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:24.113 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:10:24.113 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:10:24.144 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:10:24.145 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:10:24.176 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:10:24.176 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:10:24.209 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:24.210+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:24.220 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:24.220 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:24.220 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:24.220 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:10:24.220 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:24.232 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T20:10:24.316 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:24.316+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:24.382 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:24.383 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:24.383 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:24.383 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:24.392 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T20:10:24.434 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:24.433+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:24.512 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:24.513+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:24.541 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:24.542 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:24.542 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:24.542 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:24.542 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:24.551 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T20:10:24.613 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:24.614+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:24.663 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:24.663 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:24.664 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:24.664 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-04-25T20:10:24.664 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.664 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:30 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'16, mlcod=16'14)", 2024-04-25T20:10:24.664 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:10:24.664 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 67.494560129999996, 2024-04-25T20:10:24.664 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0054087670000000001, 2024-04-25T20:10:24.664 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:24.664 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:24.664 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:24.664 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.664 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:24.664 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:10:24.664 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.665 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.665 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.665 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:24.665 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:10:24.665 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.665 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.665 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.665 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:24.665 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167142+0000", 2024-04-25T20:10:24.665 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999976 2024-04-25T20:10:24.665 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.665 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.665 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:24.665 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167162+0000", 2024-04-25T20:10:24.666 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0109e-05 2024-04-25T20:10:24.666 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.666 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.666 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:24.666 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167188+0000", 2024-04-25T20:10:24.666 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6146e-05 2024-04-25T20:10:24.666 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.666 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.666 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:24.666 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167199+0000", 2024-04-25T20:10:24.666 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0767e-05 2024-04-25T20:10:24.666 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.666 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.667 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:24.667 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167256+0000", 2024-04-25T20:10:24.667 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7392000000000002e-05 2024-04-25T20:10:24.667 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.667 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.667 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:24.667 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167283+0000", 2024-04-25T20:10:24.667 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6577e-05 2024-04-25T20:10:24.667 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.667 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.667 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:24.667 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.172510+0000", 2024-04-25T20:10:24.667 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0052276759999999997 2024-04-25T20:10:24.667 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.668 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.668 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:24.668 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.172553+0000", 2024-04-25T20:10:24.668 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2444999999999998e-05 2024-04-25T20:10:24.668 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.668 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:24.668 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.668 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.668 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.668 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:31 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-04-25T20:10:24.668 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:24.668 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 67.493944052000003, 2024-04-25T20:10:24.668 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0055055479999999999, 2024-04-25T20:10:24.669 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:24.669 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:24.669 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:24.669 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.669 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:24.669 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:24.669 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.669 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.669 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.669 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:24.669 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:24.669 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.669 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.669 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.669 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:24.670 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:24.670 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:24.670 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.670 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.670 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:24.670 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167766+0000", 2024-04-25T20:10:24.670 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.122e-06 2024-04-25T20:10:24.670 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.670 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.670 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:24.670 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167776+0000", 2024-04-25T20:10:24.670 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0515e-05 2024-04-25T20:10:24.670 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.671 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.671 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:24.671 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167782+0000", 2024-04-25T20:10:24.671 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1149999999999999e-06 2024-04-25T20:10:24.671 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.671 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.671 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:24.671 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167837+0000", 2024-04-25T20:10:24.671 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4763999999999999e-05 2024-04-25T20:10:24.671 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.671 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.671 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:24.671 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167855+0000", 2024-04-25T20:10:24.671 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7589e-05 2024-04-25T20:10:24.671 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.672 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.672 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:24.672 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173221+0000", 2024-04-25T20:10:24.672 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.005365903 2024-04-25T20:10:24.672 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.672 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.672 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:24.672 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173266+0000", 2024-04-25T20:10:24.672 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5059999999999999e-05 2024-04-25T20:10:24.672 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.672 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:24.672 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.672 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.672 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.673 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:32 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-04-25T20:10:24.673 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:24.673 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 67.493568866999993, 2024-04-25T20:10:24.673 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0051558239999999998, 2024-04-25T20:10:24.673 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:24.673 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:24.673 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:24.673 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.673 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:24.673 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:24.673 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.673 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.673 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.673 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:24.674 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:24.674 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.674 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.674 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.674 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:24.674 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:24.674 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:24.674 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.674 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.674 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:24.674 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168173+0000", 2024-04-25T20:10:24.674 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7663999999999997e-05 2024-04-25T20:10:24.674 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.675 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.675 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:24.675 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168182+0000", 2024-04-25T20:10:24.675 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0650000000000005e-06 2024-04-25T20:10:24.675 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.675 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.675 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:24.675 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168188+0000", 2024-04-25T20:10:24.675 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0589999999999999e-06 2024-04-25T20:10:24.676 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.676 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.676 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:24.676 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168239+0000", 2024-04-25T20:10:24.676 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1467e-05 2024-04-25T20:10:24.676 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.676 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.676 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:24.676 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168260+0000", 2024-04-25T20:10:24.676 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0531e-05 2024-04-25T20:10:24.676 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.676 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.676 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:24.676 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173268+0000", 2024-04-25T20:10:24.677 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0050087980000000001 2024-04-25T20:10:24.677 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.677 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.677 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:24.677 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173291+0000", 2024-04-25T20:10:24.677 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.268e-05 2024-04-25T20:10:24.677 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.677 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:24.677 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.677 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.677 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.677 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:33 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'19, mlcod=16'14)", 2024-04-25T20:10:24.677 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:10:24.677 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 67.493058447999999, 2024-04-25T20:10:24.677 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046697049999999997, 2024-04-25T20:10:24.678 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:24.678 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:24.678 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:24.678 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.678 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:24.678 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:10:24.678 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.678 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.678 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:24.678 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:10:24.678 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.678 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.679 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:24.679 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168645+0000", 2024-04-25T20:10:24.679 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:24.679 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.679 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.679 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:24.679 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168651+0000", 2024-04-25T20:10:24.679 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1079999999999998e-06 2024-04-25T20:10:24.679 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.679 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.679 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:24.679 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168660+0000", 2024-04-25T20:10:24.679 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0010000000000007e-06 2024-04-25T20:10:24.679 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.679 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.679 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:24.679 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168666+0000", 2024-04-25T20:10:24.680 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7780000000000004e-06 2024-04-25T20:10:24.680 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.680 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.680 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:24.680 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168701+0000", 2024-04-25T20:10:24.680 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4490999999999999e-05 2024-04-25T20:10:24.680 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.680 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.680 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:24.680 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168729+0000", 2024-04-25T20:10:24.680 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8476999999999999e-05 2024-04-25T20:10:24.680 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.680 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.680 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:24.680 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173294+0000", 2024-04-25T20:10:24.681 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0045650730000000002 2024-04-25T20:10:24.681 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.681 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.681 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:24.681 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173315+0000", 2024-04-25T20:10:24.681 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1245000000000001e-05 2024-04-25T20:10:24.681 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.681 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:24.681 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.681 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.681 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.681 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:34 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'20, mlcod=16'14)", 2024-04-25T20:10:24.681 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:10:24.681 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 67.492559518999997, 2024-04-25T20:10:24.681 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041943309999999999, 2024-04-25T20:10:24.682 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:24.682 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:24.682 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:24.682 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.682 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:24.682 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:10:24.682 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.682 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.682 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:24.682 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:10:24.682 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.682 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.682 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:24.683 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169144+0000", 2024-04-25T20:10:24.683 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:24.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.683 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.683 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:24.683 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169150+0000", 2024-04-25T20:10:24.683 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9150000000000001e-06 2024-04-25T20:10:24.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.683 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.683 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:24.683 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169159+0000", 2024-04-25T20:10:24.683 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8899999999999996e-06 2024-04-25T20:10:24.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.683 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.683 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:24.684 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169169+0000", 2024-04-25T20:10:24.684 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0441e-05 2024-04-25T20:10:24.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.684 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.684 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:24.684 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169206+0000", 2024-04-25T20:10:24.684 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6816000000000001e-05 2024-04-25T20:10:24.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.684 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.684 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:24.684 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169234+0000", 2024-04-25T20:10:24.684 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8008999999999999e-05 2024-04-25T20:10:24.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.684 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.684 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:24.685 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173317+0000", 2024-04-25T20:10:24.685 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0040830160000000001 2024-04-25T20:10:24.685 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.685 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.685 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:24.685 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173339+0000", 2024-04-25T20:10:24.685 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1897000000000001e-05 2024-04-25T20:10:24.685 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.685 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:24.685 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.685 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.685 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.685 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:35 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'14)", 2024-04-25T20:10:24.685 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:10:24.685 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 67.492067199999994, 2024-04-25T20:10:24.686 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037236529999999999, 2024-04-25T20:10:24.686 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:24.686 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:24.686 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:24.686 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.686 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:24.686 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:10:24.686 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.686 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.686 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.686 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:24.686 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:10:24.686 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.686 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.686 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.687 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:24.687 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169636+0000", 2024-04-25T20:10:24.687 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:24.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.687 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.687 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:24.687 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169678+0000", 2024-04-25T20:10:24.687 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1752000000000003e-05 2024-04-25T20:10:24.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.687 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.687 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:24.687 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169688+0000", 2024-04-25T20:10:24.687 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0098000000000001e-05 2024-04-25T20:10:24.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.687 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.687 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:24.688 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169694+0000", 2024-04-25T20:10:24.688 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2349999999999996e-06 2024-04-25T20:10:24.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.688 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.688 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:24.688 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169733+0000", 2024-04-25T20:10:24.688 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8481e-05 2024-04-25T20:10:24.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.688 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.688 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:24.688 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169750+0000", 2024-04-25T20:10:24.688 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7509000000000001e-05 2024-04-25T20:10:24.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.688 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.688 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:24.689 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173340+0000", 2024-04-25T20:10:24.689 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0035898449999999999 2024-04-25T20:10:24.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.689 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.689 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:24.689 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173361+0000", 2024-04-25T20:10:24.689 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.039e-05 2024-04-25T20:10:24.689 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.689 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:24.689 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.689 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.689 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:36 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'22, mlcod=16'14)", 2024-04-25T20:10:24.689 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:24.689 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 67.491567140000001, 2024-04-25T20:10:24.689 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032472120000000002, 2024-04-25T20:10:24.690 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:24.690 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:24.690 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:24.690 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.690 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:24.690 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:24.690 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.690 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.690 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:24.690 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:24.690 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.690 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.691 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:24.691 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:24.691 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:24.691 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.691 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.691 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:24.691 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170152+0000", 2024-04-25T20:10:24.691 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5798e-05 2024-04-25T20:10:24.691 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.691 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.691 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:24.691 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170161+0000", 2024-04-25T20:10:24.691 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.064e-06 2024-04-25T20:10:24.691 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.691 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.692 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:24.692 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170167+0000", 2024-04-25T20:10:24.692 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8390000000000003e-06 2024-04-25T20:10:24.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.692 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.692 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:24.692 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170202+0000", 2024-04-25T20:10:24.692 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.5120999999999998e-05 2024-04-25T20:10:24.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.692 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.692 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:24.692 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170219+0000", 2024-04-25T20:10:24.692 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6929999999999999e-05 2024-04-25T20:10:24.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.693 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.693 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:24.693 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173362+0000", 2024-04-25T20:10:24.693 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0031427619999999999 2024-04-25T20:10:24.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.693 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.693 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:24.693 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173384+0000", 2024-04-25T20:10:24.693 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2237000000000001e-05 2024-04-25T20:10:24.693 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.693 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:24.693 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.693 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.694 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:40 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'26, mlcod=16'24)", 2024-04-25T20:10:24.694 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:10:24.694 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 67.484529228, 2024-04-25T20:10:24.694 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028727330000000001, 2024-04-25T20:10:24.694 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:24.694 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:24.694 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:24.694 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.694 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:24.694 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:10:24.694 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.694 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.694 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.695 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:24.695 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:10:24.695 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.695 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.695 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:24.695 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177173+0000", 2024-04-25T20:10:24.695 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999976 2024-04-25T20:10:24.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.695 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.695 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:24.695 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177186+0000", 2024-04-25T20:10:24.695 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3533000000000001e-05 2024-04-25T20:10:24.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.695 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.695 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:24.696 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177209+0000", 2024-04-25T20:10:24.696 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3280000000000001e-05 2024-04-25T20:10:24.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.696 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.696 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:24.696 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177220+0000", 2024-04-25T20:10:24.696 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0465000000000001e-05 2024-04-25T20:10:24.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.696 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.696 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:24.696 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177264+0000", 2024-04-25T20:10:24.696 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4413999999999999e-05 2024-04-25T20:10:24.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.696 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.696 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:24.697 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177289+0000", 2024-04-25T20:10:24.697 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4783999999999999e-05 2024-04-25T20:10:24.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.697 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.697 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:24.697 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.179990+0000", 2024-04-25T20:10:24.697 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027012939999999999 2024-04-25T20:10:24.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.697 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.697 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:24.697 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.180048+0000", 2024-04-25T20:10:24.697 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7299000000000001e-05 2024-04-25T20:10:24.697 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.697 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:24.697 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.698 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.698 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:67 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'41, mlcod=16'39)", 2024-04-25T20:10:24.698 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:10:24.698 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 67.449767735999998, 2024-04-25T20:10:24.698 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064192549999999996, 2024-04-25T20:10:24.698 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:24.698 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:24.698 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:24.698 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.698 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:24.698 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:10:24.698 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.699 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.699 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:24.699 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:10:24.699 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.699 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.699 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.699 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:24.699 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211935+0000", 2024-04-25T20:10:24.699 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999981 2024-04-25T20:10:24.699 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.699 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.699 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:24.699 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211948+0000", 2024-04-25T20:10:24.699 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3531e-05 2024-04-25T20:10:24.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.700 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.700 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:24.700 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211967+0000", 2024-04-25T20:10:24.700 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8326999999999999e-05 2024-04-25T20:10:24.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.700 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.700 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:24.700 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211975+0000", 2024-04-25T20:10:24.700 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8589999999999994e-06 2024-04-25T20:10:24.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.700 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.700 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:24.700 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212020+0000", 2024-04-25T20:10:24.701 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4626999999999998e-05 2024-04-25T20:10:24.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.701 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.701 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:24.701 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212045+0000", 2024-04-25T20:10:24.701 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4508999999999999e-05 2024-04-25T20:10:24.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.701 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.701 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:24.701 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218316+0000", 2024-04-25T20:10:24.701 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0062711640000000001 2024-04-25T20:10:24.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.701 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.701 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:24.701 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218356+0000", 2024-04-25T20:10:24.702 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9993000000000001e-05 2024-04-25T20:10:24.702 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.702 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:24.702 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.702 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.702 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:68 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-04-25T20:10:24.702 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:24.702 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 67.449318722000001, 2024-04-25T20:10:24.702 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011231365, 2024-04-25T20:10:24.702 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:24.702 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:24.702 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:24.702 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.703 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:24.703 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:24.703 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.703 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.703 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:24.703 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:24.703 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.703 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.703 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:24.703 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:24.703 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:24.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.703 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.704 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:24.704 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212392+0000", 2024-04-25T20:10:24.705 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.79e-06 2024-04-25T20:10:24.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.705 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.705 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:24.705 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212406+0000", 2024-04-25T20:10:24.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3577000000000001e-05 2024-04-25T20:10:24.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.706 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:24.706 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212414+0000", 2024-04-25T20:10:24.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6510000000000001e-06 2024-04-25T20:10:24.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.706 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:24.706 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217114+0000", 2024-04-25T20:10:24.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047005379999999998 2024-04-25T20:10:24.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.706 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:24.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217141+0000", 2024-04-25T20:10:24.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6984000000000001e-05 2024-04-25T20:10:24.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:24.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223563+0000", 2024-04-25T20:10:24.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064216999999999998 2024-04-25T20:10:24.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.708 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:24.708 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223617+0000", 2024-04-25T20:10:24.708 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4011000000000002e-05 2024-04-25T20:10:24.708 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.708 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:24.708 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.709 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:69 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-04-25T20:10:24.709 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:10:24.709 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 67.448786532, 2024-04-25T20:10:24.709 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.010730094000000001, 2024-04-25T20:10:24.709 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:24.709 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:24.709 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:24.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.709 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:24.709 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:10:24.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:24.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:10:24.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:24.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212917+0000", 2024-04-25T20:10:24.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:24.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.711 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:24.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212957+0000", 2024-04-25T20:10:24.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0692000000000003e-05 2024-04-25T20:10:24.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.711 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:24.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212973+0000", 2024-04-25T20:10:24.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.575e-05 2024-04-25T20:10:24.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.711 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:24.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212980+0000", 2024-04-25T20:10:24.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1679999999999998e-06 2024-04-25T20:10:24.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.712 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.712 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:24.712 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217449+0000", 2024-04-25T20:10:24.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.004468367 2024-04-25T20:10:24.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.712 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.712 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:24.712 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217470+0000", 2024-04-25T20:10:24.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0749000000000001e-05 2024-04-25T20:10:24.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.712 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.712 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:24.712 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223621+0000", 2024-04-25T20:10:24.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0061512880000000004 2024-04-25T20:10:24.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.713 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:24.713 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223648+0000", 2024-04-25T20:10:24.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6961000000000002e-05 2024-04-25T20:10:24.713 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.713 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:24.713 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.713 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:70 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'44, mlcod=16'39)", 2024-04-25T20:10:24.713 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:24.713 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 67.448027804000006, 2024-04-25T20:10:24.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.013553358, 2024-04-25T20:10:24.713 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:24.714 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:24.714 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:24.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:24.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:24.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:24.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:24.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:24.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:24.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:24.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:24.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213682+0000", 2024-04-25T20:10:24.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3439999999999997e-06 2024-04-25T20:10:24.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:24.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213693+0000", 2024-04-25T20:10:24.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1049999999999999e-05 2024-04-25T20:10:24.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:24.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213699+0000", 2024-04-25T20:10:24.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9399999999999999e-06 2024-04-25T20:10:24.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:24.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218086+0000", 2024-04-25T20:10:24.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0043871240000000001 2024-04-25T20:10:24.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:24.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218106+0000", 2024-04-25T20:10:24.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.997e-05 2024-04-25T20:10:24.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.717 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:24.717 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.227186+0000", 2024-04-25T20:10:24.717 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.009079574 2024-04-25T20:10:24.717 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.717 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.717 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:24.717 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.227230+0000", 2024-04-25T20:10:24.717 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4147999999999998e-05 2024-04-25T20:10:24.717 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.717 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:24.717 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.717 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.717 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.717 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:106 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-04-25T20:10:24.717 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:10:24.718 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 67.076888703999998, 2024-04-25T20:10:24.718 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002936756, 2024-04-25T20:10:24.718 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:24.718 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:24.718 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:24.718 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.718 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:24.718 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:10:24.718 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:24.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:10:24.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:24.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584815+0000", 2024-04-25T20:10:24.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:24.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:24.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584860+0000", 2024-04-25T20:10:24.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5161999999999999e-05 2024-04-25T20:10:24.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:24.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584874+0000", 2024-04-25T20:10:24.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4464000000000001e-05 2024-04-25T20:10:24.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.721 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.721 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:24.721 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584881+0000", 2024-04-25T20:10:24.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.409e-06 2024-04-25T20:10:24.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.721 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.721 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:24.721 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584915+0000", 2024-04-25T20:10:24.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4007000000000002e-05 2024-04-25T20:10:24.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.721 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.721 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:24.721 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584933+0000", 2024-04-25T20:10:24.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7737999999999999e-05 2024-04-25T20:10:24.722 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.722 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.722 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:24.722 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587717+0000", 2024-04-25T20:10:24.722 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002784711 2024-04-25T20:10:24.722 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.722 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.722 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:24.722 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587752+0000", 2024-04-25T20:10:24.722 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4915e-05 2024-04-25T20:10:24.722 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.722 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:24.722 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.723 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:107 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'69, mlcod=16'64)", 2024-04-25T20:10:24.723 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:10:24.723 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 67.076813401999999, 2024-04-25T20:10:24.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028906270000000002, 2024-04-25T20:10:24.723 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:24.723 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:24.723 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:24.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:24.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:10:24.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:24.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:10:24.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:24.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584890+0000", 2024-04-25T20:10:24.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:24.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:24.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584895+0000", 2024-04-25T20:10:24.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2069999999999997e-06 2024-04-25T20:10:24.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:24.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584906+0000", 2024-04-25T20:10:24.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.117e-05 2024-04-25T20:10:24.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:24.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584910+0000", 2024-04-25T20:10:24.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4190000000000004e-06 2024-04-25T20:10:24.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:24.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585310+0000", 2024-04-25T20:10:24.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00040002700000000001 2024-04-25T20:10:24.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:24.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585329+0000", 2024-04-25T20:10:24.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8838000000000002e-05 2024-04-25T20:10:24.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.726 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:24.726 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587754+0000", 2024-04-25T20:10:24.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024252689999999999 2024-04-25T20:10:24.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.726 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:24.726 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587781+0000", 2024-04-25T20:10:24.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7206999999999999e-05 2024-04-25T20:10:24.726 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.726 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:24.726 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.727 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:108 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'70, mlcod=16'64)", 2024-04-25T20:10:24.727 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:10:24.727 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 67.076788250999996, 2024-04-25T20:10:24.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028924879999999999, 2024-04-25T20:10:24.727 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:24.727 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:24.727 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:24.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.727 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:24.727 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:10:24.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.727 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:24.727 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:10:24.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:24.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:10:24.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:24.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:24.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584919+0000", 2024-04-25T20:10:24.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.557e-06 2024-04-25T20:10:24.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:24.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584925+0000", 2024-04-25T20:10:24.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8619999999999999e-06 2024-04-25T20:10:24.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:24.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584931+0000", 2024-04-25T20:10:24.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2539999999999998e-06 2024-04-25T20:10:24.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:24.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585550+0000", 2024-04-25T20:10:24.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00061821800000000002 2024-04-25T20:10:24.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:24.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585567+0000", 2024-04-25T20:10:24.730 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7499e-05 2024-04-25T20:10:24.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.730 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.730 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:24.730 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587783+0000", 2024-04-25T20:10:24.730 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0022156200000000002 2024-04-25T20:10:24.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.730 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.730 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:24.730 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587808+0000", 2024-04-25T20:10:24.730 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5729e-05 2024-04-25T20:10:24.730 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.730 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:24.730 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.731 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.731 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:112 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'74, mlcod=16'66)", 2024-04-25T20:10:24.731 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:10:24.731 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 67.074630628999998, 2024-04-25T20:10:24.731 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00318724, 2024-04-25T20:10:24.731 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:24.731 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:24.731 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:24.731 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.731 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:24.731 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:10:24.731 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.731 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.731 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:24.732 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:10:24.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.732 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:24.732 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587073+0000", 2024-04-25T20:10:24.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:24.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.732 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:24.732 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587136+0000", 2024-04-25T20:10:24.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2928999999999993e-05 2024-04-25T20:10:24.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.732 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:24.732 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587147+0000", 2024-04-25T20:10:24.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1182999999999999e-05 2024-04-25T20:10:24.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:24.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587153+0000", 2024-04-25T20:10:24.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9059999999999999e-06 2024-04-25T20:10:24.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:24.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587306+0000", 2024-04-25T20:10:24.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000152848 2024-04-25T20:10:24.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:24.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587330+0000", 2024-04-25T20:10:24.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4304e-05 2024-04-25T20:10:24.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:24.734 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.590230+0000", 2024-04-25T20:10:24.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029004629999999998 2024-04-25T20:10:24.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:24.734 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.590261+0000", 2024-04-25T20:10:24.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0323000000000001e-05 2024-04-25T20:10:24.734 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.734 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:24.734 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.734 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:118 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'80, mlcod=16'78)", 2024-04-25T20:10:24.735 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:10:24.735 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 67.06984405, 2024-04-25T20:10:24.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029618959999999999, 2024-04-25T20:10:24.735 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:24.735 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:24.735 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:24.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:24.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:10:24.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:24.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:10:24.736 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.736 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:24.736 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591859+0000", 2024-04-25T20:10:24.736 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:24.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.736 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:24.736 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591918+0000", 2024-04-25T20:10:24.736 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8842999999999999e-05 2024-04-25T20:10:24.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.736 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:24.736 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591933+0000", 2024-04-25T20:10:24.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4600000000000001e-05 2024-04-25T20:10:24.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:24.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591939+0000", 2024-04-25T20:10:24.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4559999999999997e-06 2024-04-25T20:10:24.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:24.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591990+0000", 2024-04-25T20:10:24.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0318999999999997e-05 2024-04-25T20:10:24.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:24.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.592008+0000", 2024-04-25T20:10:24.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8819000000000001e-05 2024-04-25T20:10:24.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.738 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:24.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.594784+0000", 2024-04-25T20:10:24.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027750850000000001 2024-04-25T20:10:24.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.738 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:24.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.594822+0000", 2024-04-25T20:10:24.739 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8522000000000001e-05 2024-04-25T20:10:24.739 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.739 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:24.739 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.739 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:139 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'97, mlcod=17'87)", 2024-04-25T20:10:24.739 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:10:24.739 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 66.535807445000003, 2024-04-25T20:10:24.739 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048546290000000001, 2024-04-25T20:10:24.739 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:24.739 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:24.739 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:24.740 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.740 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:24.740 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:10:24.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.740 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.740 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:24.740 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:10:24.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.740 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.740 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:24.740 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125896+0000", 2024-04-25T20:10:24.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:24.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.740 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.741 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:24.741 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125908+0000", 2024-04-25T20:10:24.741 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.137e-05 2024-04-25T20:10:24.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.741 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:24.741 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125915+0000", 2024-04-25T20:10:24.741 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3939999999999998e-06 2024-04-25T20:10:24.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.741 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:24.741 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125922+0000", 2024-04-25T20:10:24.741 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1930000000000004e-06 2024-04-25T20:10:24.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.741 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:24.741 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125959+0000", 2024-04-25T20:10:24.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6701e-05 2024-04-25T20:10:24.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:24.742 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125972+0000", 2024-04-25T20:10:24.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2578000000000001e-05 2024-04-25T20:10:24.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:24.742 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130719+0000", 2024-04-25T20:10:24.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047475800000000004 2024-04-25T20:10:24.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.743 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:24.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130751+0000", 2024-04-25T20:10:24.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2283000000000002e-05 2024-04-25T20:10:24.743 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.743 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:24.743 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.743 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.743 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:140 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'98, mlcod=17'87)", 2024-04-25T20:10:24.743 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:24.743 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 66.535382093999999, 2024-04-25T20:10:24.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048310990000000002, 2024-04-25T20:10:24.743 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:24.743 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:24.743 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:24.744 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.744 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:24.744 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:24.744 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.744 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.744 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.744 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:24.744 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:24.744 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.744 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.744 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.744 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:24.744 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:24.744 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:24.744 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.744 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.745 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:24.745 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126427+0000", 2024-04-25T20:10:24.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000105572 2024-04-25T20:10:24.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.745 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:24.745 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126435+0000", 2024-04-25T20:10:24.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1920000000000005e-06 2024-04-25T20:10:24.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.745 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:24.745 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126439+0000", 2024-04-25T20:10:24.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6950000000000002e-06 2024-04-25T20:10:24.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.745 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:24.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126471+0000", 2024-04-25T20:10:24.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1699000000000003e-05 2024-04-25T20:10:24.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:24.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126484+0000", 2024-04-25T20:10:24.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.343e-05 2024-04-25T20:10:24.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:24.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131131+0000", 2024-04-25T20:10:24.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046471480000000003 2024-04-25T20:10:24.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.747 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:24.747 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131153+0000", 2024-04-25T20:10:24.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1753e-05 2024-04-25T20:10:24.747 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.747 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:24.747 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.747 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:141 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'99, mlcod=17'87)", 2024-04-25T20:10:24.747 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:24.747 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 66.534727149999995, 2024-04-25T20:10:24.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.004496847, 2024-04-25T20:10:24.748 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:24.748 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:24.748 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:24.748 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.748 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:24.748 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:24.748 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.748 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.748 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:24.748 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:24.748 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:24.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.748 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.749 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:24.749 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:24.749 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:24.749 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.749 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.749 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:24.749 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126980+0000", 2024-04-25T20:10:24.749 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.619e-06 2024-04-25T20:10:24.749 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.749 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.749 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:24.749 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126987+0000", 2024-04-25T20:10:24.749 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7220000000000002e-06 2024-04-25T20:10:24.749 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.749 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.750 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:24.750 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126992+0000", 2024-04-25T20:10:24.750 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8679999999999998e-06 2024-04-25T20:10:24.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.750 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.750 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:24.750 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130772+0000", 2024-04-25T20:10:24.750 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037803120000000001 2024-04-25T20:10:24.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.750 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.750 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:24.750 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130786+0000", 2024-04-25T20:10:24.750 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3592e-05 2024-04-25T20:10:24.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.751 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:24.751 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131451+0000", 2024-04-25T20:10:24.751 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000665206 2024-04-25T20:10:24.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:24.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:24.751 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:24.751 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131474+0000", 2024-04-25T20:10:24.751 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2957999999999999e-05 2024-04-25T20:10:24.751 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.751 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:24.751 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.751 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:24.751 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:24.751 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:24.752 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is unset 2024-04-25T20:10:24.753 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T20:10:24.754 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:24.752+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:24.854 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:24.853+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:24.905 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:24.905 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:24.905 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:24.905 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:24.915 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T20:10:24.955 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:24.955+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:25.055 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:25.055+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:25.061 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:25.061 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:25.061 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:25.062 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:25.062 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:25.071 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T20:10:25.155 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:25.155+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:25.177 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:25.178 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:25.178 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:25.178 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:10:25.178 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:25.187 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T20:10:25.256 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:25.255+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:25.336 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:25.336 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:25.336 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:25.336 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:25.345 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T20:10:25.357 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:25.356+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:25.453 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:25.453 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:25.454 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:25.454 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:25.454 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:25.457 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:25.457+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:25.463 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T20:10:25.559 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:25.558+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:25.614 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:25.615 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:25.615 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:25.615 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:10:25.615 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:25.624 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T20:10:25.669 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:25.668+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:25.738 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:25.738 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:25.739 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:10:25.739 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:25.748 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T20:10:25.760 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:25.760+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:25.770 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7] out_osds: [3, 2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-25T20:10:25.770 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T20:10:25.771 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-04-25T20:10:25.771 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 0.978680 2024-04-25T20:10:25.771 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 0.9786798052696676 2024-04-25T20:10:25.861 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:25.861+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:25.890 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:25.890 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:25.890 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:25.890 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:25.890 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:25.899 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T20:10:25.962 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:25.961+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:26.001 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:26.001 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:26.001 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:26.001 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:10:26.001 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:26.009 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T20:10:26.094 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:26.093+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:26.152 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:26.153 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:26.153 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:10:26.153 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:26.161 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T20:10:26.163 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:26.162+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:26.192 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T20:10:26.263 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:26.263+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:26.303 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:26.303 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:26.303 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:26.303 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:26.303 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:26.311 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T20:10:26.365 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:26.364+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:26.380 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:10:26.380 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:10:26.399 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:10:26.399 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:10:26.413 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:26.413 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:26.413 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:26.413 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:10:26.413 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:26.422 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T20:10:26.431 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:10:26.431 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:10:26.449 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:10:26.449 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:10:26.466 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:26.465+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:26.480 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:10:26.480 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:10:26.505 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:10:26.505 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:10:26.522 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:10:26.522 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:10:26.524 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:26.524 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:26.533 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T20:10:26.539 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:10:26.539 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:10:26.594 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:26.594+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:26.666 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:26.666+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:26.674 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:26.674 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:26.683 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T20:10:26.686 INFO:teuthology.orchestra.run.smithi039.stderr:set osd.6 primary-affinity to 0.97868 (8641382) 2024-04-25T20:10:26.756 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-25T20:10:26.783 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:26.783 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:26.791 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T20:10:26.799 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:26.799+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:26.869 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:26.868+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:26.894 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:26.894 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:26.902 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T20:10:26.969 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:26.969+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:27.004 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:27.004 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:27.013 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T20:10:27.070 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:27.070+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:27.154 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:27.155 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:27.164 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T20:10:27.208 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:27.208+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:27.272 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:27.272+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:27.275 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:27.275 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:27.284 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T20:10:27.373 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:27.372+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:27.447 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:27.447 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:27.459 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T20:10:27.503 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:27.503+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:27.565 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:27.565 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:27.574 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T20:10:27.575 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:27.574+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:27.675 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:27.674+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:27.712 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is set 2024-04-25T20:10:27.750 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:27.750 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:27.763 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T20:10:27.776 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:27.776+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:27.876 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:27.876+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:27.909 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:27.909 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:27.919 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T20:10:27.977 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:27.977+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:28.026 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:28.026 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:28.035 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T20:10:28.080 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:28.080+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:28.142 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:28.142 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:28.152 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T20:10:28.178 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:28.177+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:28.279 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:28.278+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:28.299 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:28.299 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:28.309 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T20:10:28.394 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:28.394+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:28.458 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:28.458 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:28.468 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T20:10:28.481 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:28.481+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:28.554 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T20:10:28.582 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:28.581+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:28.617 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:28.617 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:28.627 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T20:10:28.683 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:28.683+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:28.687 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:10:28.687 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:10:28.706 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:10:28.707 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:10:28.725 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:10:28.725 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:10:28.739 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:28.739 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:28.742 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:10:28.743 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:10:28.750 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T20:10:28.761 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:10:28.762 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:10:28.779 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:10:28.780 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:10:28.784 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:28.783+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:28.799 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:10:28.799 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:10:28.816 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:10:28.817 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:10:28.872 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:28.872 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:28.872 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:28.872 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:10:28.873 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:28.882 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T20:10:28.885 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:28.884+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:28.985 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:28.985+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:28.995 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:28.995 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:28.995 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:10:28.995 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:29.004 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T20:10:29.086 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:29.086+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:29.148 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:29.148 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:29.148 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:29.148 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:29.148 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:29.159 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T20:10:29.203 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:29.202+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:29.261 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:29.261 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:29.261 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:29.261 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:10:29.261 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:29.269 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T20:10:29.311 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:29.310+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:29.388 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:29.388+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:29.412 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:29.412 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:29.412 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:10:29.412 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:29.421 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T20:10:29.490 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:29.489+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:29.562 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:29.562 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:29.562 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:29.562 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:29.562 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:29.571 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T20:10:29.615 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:29.614+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:29.674 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:29.674 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:29.674 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:29.674 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:10:29.674 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:29.682 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T20:10:29.692 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:29.691+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:29.730 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-25T20:10:29.793 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:29.792+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:29.824 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:29.825 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:29.825 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:10:29.825 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:29.834 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T20:10:29.918 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:29.917+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:29.975 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:29.975 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:29.975 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:29.975 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:29.975 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:29.984 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T20:10:29.994 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:29.994+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:30.095 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:30.095+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:30.126 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:30.126 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:30.126 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:30.126 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:10:30.126 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:30.135 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T20:10:30.196 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:30.196+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:30.277 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:30.277 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:30.277 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:10:30.278 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:30.286 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T20:10:30.297 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:30.297+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:30.387 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:30.387 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:30.387 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:30.387 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:30.387 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:30.396 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T20:10:30.398 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:30.397+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:30.500 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:30.500 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:30.500 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:30.500 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [ 2024-04-25T20:10:30.500 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.500 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:30.500 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:10:30.501 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 73.715895591000006, 2024-04-25T20:10:30.501 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.34154860100000001, 2024-04-25T20:10:30.501 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:30.501 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:30.501 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:30.501 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:30.501 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:30.501 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 1 2024-04-25T20:10:30.501 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.501 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:30.501 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.501 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:30.501 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:10:30.502 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.502 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.502 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.502 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:30.502 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:10:30.502 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.502 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.502 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.502 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:30.502 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782261+0000", 2024-04-25T20:10:30.502 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999986 2024-04-25T20:10:30.502 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.502 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.502 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:30.502 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782268+0000", 2024-04-25T20:10:30.502 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.8140000000000001e-06 2024-04-25T20:10:30.503 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.503 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.503 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:30.503 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782279+0000", 2024-04-25T20:10:30.503 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0665999999999999e-05 2024-04-25T20:10:30.503 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.503 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.503 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:30.503 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782290+0000", 2024-04-25T20:10:30.503 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.1343e-05 2024-04-25T20:10:30.503 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.503 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.503 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:30.503 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782340+0000", 2024-04-25T20:10:30.503 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.9311999999999998e-05 2024-04-25T20:10:30.503 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.504 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.504 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for peered", 2024-04-25T20:10:30.504 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782348+0000", 2024-04-25T20:10:30.504 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.0690000000000002e-06 2024-04-25T20:10:30.504 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.504 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.504 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:30.504 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120520+0000", 2024-04-25T20:10:30.504 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.33817188100000001 2024-04-25T20:10:30.504 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.504 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.504 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:30.504 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120656+0000", 2024-04-25T20:10:30.504 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000135897 2024-04-25T20:10:30.504 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.504 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.505 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:30.505 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120802+0000", 2024-04-25T20:10:30.505 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00014691199999999999 2024-04-25T20:10:30.505 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.505 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.505 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:30.505 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.122368+0000", 2024-04-25T20:10:30.505 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0015651040000000001 2024-04-25T20:10:30.505 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.505 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.505 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:30.505 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123729+0000", 2024-04-25T20:10:30.505 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0013610499999999999 2024-04-25T20:10:30.505 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.505 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.505 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:30.505 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123769+0000", 2024-04-25T20:10:30.506 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0719000000000001e-05 2024-04-25T20:10:30.506 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.506 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.506 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:30.506 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123811+0000", 2024-04-25T20:10:30.506 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1267999999999999e-05 2024-04-25T20:10:30.506 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.506 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:30.506 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.506 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.506 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.506 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:67 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:30.506 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:10:30.506 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 73.286476703999995, 2024-04-25T20:10:30.506 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.006947651, 2024-04-25T20:10:30.507 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:30.507 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:30.507 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:30.507 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:30.507 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:30.507 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 67 2024-04-25T20:10:30.507 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.507 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:30.507 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.507 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:30.507 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:10:30.507 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.507 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.507 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.508 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:30.508 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:10:30.508 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.508 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.508 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.508 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:30.508 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211680+0000", 2024-04-25T20:10:30.508 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:30.508 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.508 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.508 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:30.508 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211684+0000", 2024-04-25T20:10:30.508 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4249999999999998e-06 2024-04-25T20:10:30.508 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.508 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.508 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:30.509 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211690+0000", 2024-04-25T20:10:30.509 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.7360000000000004e-06 2024-04-25T20:10:30.509 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.509 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.509 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:30.509 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211694+0000", 2024-04-25T20:10:30.509 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4860000000000002e-06 2024-04-25T20:10:30.509 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.509 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.509 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:30.509 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211727+0000", 2024-04-25T20:10:30.509 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3445999999999998e-05 2024-04-25T20:10:30.509 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.510 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.510 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:30.510 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211778+0000", 2024-04-25T20:10:30.510 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1050000000000001e-05 2024-04-25T20:10:30.510 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.510 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.510 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:30.510 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211851+0000", 2024-04-25T20:10:30.510 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2914e-05 2024-04-25T20:10:30.510 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.510 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.510 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:30.510 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213824+0000", 2024-04-25T20:10:30.511 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0019726520000000001 2024-04-25T20:10:30.511 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.511 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.511 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:30.511 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218579+0000", 2024-04-25T20:10:30.511 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0047553159999999999 2024-04-25T20:10:30.511 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.511 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.511 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:30.511 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218607+0000", 2024-04-25T20:10:30.511 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8144999999999999e-05 2024-04-25T20:10:30.511 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.511 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.511 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:30.512 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218629+0000", 2024-04-25T20:10:30.512 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.1044e-05 2024-04-25T20:10:30.512 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.512 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:30.512 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.512 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.512 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.512 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:68 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:30.512 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:30.512 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 73.286458362999994, 2024-04-25T20:10:30.512 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012255596000000001, 2024-04-25T20:10:30.512 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:30.512 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:30.512 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:30.513 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:30.513 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:30.513 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 68 2024-04-25T20:10:30.513 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.513 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:30.513 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.513 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:30.513 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:30.513 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.513 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.513 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.513 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:30.513 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:30.513 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.513 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.514 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.514 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:30.514 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:30.514 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:30.514 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.514 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.514 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:30.514 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211701+0000", 2024-04-25T20:10:30.514 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2029999999999999e-06 2024-04-25T20:10:30.514 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.514 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.514 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:30.514 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211705+0000", 2024-04-25T20:10:30.514 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.2320000000000003e-06 2024-04-25T20:10:30.514 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.514 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.515 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:30.515 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211708+0000", 2024-04-25T20:10:30.515 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.6290000000000002e-06 2024-04-25T20:10:30.515 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.515 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.515 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:30.515 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212136+0000", 2024-04-25T20:10:30.515 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00042764099999999998 2024-04-25T20:10:30.515 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.515 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.515 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:30.515 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212212+0000", 2024-04-25T20:10:30.515 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.6315000000000004e-05 2024-04-25T20:10:30.515 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.515 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.515 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:30.516 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212306+0000", 2024-04-25T20:10:30.516 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.3556999999999999e-05 2024-04-25T20:10:30.516 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.516 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.516 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:30.516 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213829+0000", 2024-04-25T20:10:30.516 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001523256 2024-04-25T20:10:30.516 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.516 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.516 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:30.516 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223904+0000", 2024-04-25T20:10:30.516 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010075387 2024-04-25T20:10:30.516 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.516 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.516 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:30.517 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223934+0000", 2024-04-25T20:10:30.517 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9988000000000002e-05 2024-04-25T20:10:30.517 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.517 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.517 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:30.517 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223955+0000", 2024-04-25T20:10:30.517 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0630999999999999e-05 2024-04-25T20:10:30.517 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.517 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:30.517 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.517 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.517 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.518 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:69 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:30.518 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:30.518 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 73.286436628000004, 2024-04-25T20:10:30.518 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012316863000000001, 2024-04-25T20:10:30.518 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:30.518 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:30.518 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:30.518 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:30.518 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:30.518 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 69 2024-04-25T20:10:30.518 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.518 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:30.518 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.519 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:30.519 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:30.519 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.519 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.519 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.519 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:30.519 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:30.519 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.519 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.519 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.519 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:30.519 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:30.519 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:30.519 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.519 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.520 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:30.520 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211762+0000", 2024-04-25T20:10:30.520 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1703999999999999e-05 2024-04-25T20:10:30.520 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.520 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.520 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:30.520 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211768+0000", 2024-04-25T20:10:30.520 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.3430000000000003e-06 2024-04-25T20:10:30.520 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.520 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.520 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:30.520 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211771+0000", 2024-04-25T20:10:30.520 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.18e-06 2024-04-25T20:10:30.520 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.520 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.521 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:30.521 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212652+0000", 2024-04-25T20:10:30.521 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00088095800000000004 2024-04-25T20:10:30.521 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.521 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.521 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:30.521 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212703+0000", 2024-04-25T20:10:30.521 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.0655999999999997e-05 2024-04-25T20:10:30.521 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.521 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.521 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:30.521 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212781+0000", 2024-04-25T20:10:30.521 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.8852999999999999e-05 2024-04-25T20:10:30.521 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.521 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.522 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:30.522 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.214131+0000", 2024-04-25T20:10:30.522 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0013496039999999999 2024-04-25T20:10:30.522 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.522 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.522 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:30.522 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223986+0000", 2024-04-25T20:10:30.522 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.009854804 2024-04-25T20:10:30.522 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.522 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.522 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:30.523 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.224017+0000", 2024-04-25T20:10:30.523 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.1068999999999997e-05 2024-04-25T20:10:30.523 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.523 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.523 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:30.523 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.224038+0000", 2024-04-25T20:10:30.523 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0965e-05 2024-04-25T20:10:30.523 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.523 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:30.523 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.523 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.523 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.523 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:70 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:30.523 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:30.523 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 73.286381547000005, 2024-04-25T20:10:30.523 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.015776348999999999, 2024-04-25T20:10:30.524 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:30.524 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:30.524 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:30.524 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:30.524 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:30.524 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 70 2024-04-25T20:10:30.524 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.524 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:30.524 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.524 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:30.524 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:30.524 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.524 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.524 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.524 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:30.524 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:30.525 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.525 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.525 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.525 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:30.525 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:30.525 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:30.525 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.525 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.525 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:30.525 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211778+0000", 2024-04-25T20:10:30.525 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5330000000000001e-06 2024-04-25T20:10:30.525 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.525 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.525 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:30.525 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211782+0000", 2024-04-25T20:10:30.525 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4079999999999998e-06 2024-04-25T20:10:30.526 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.526 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.526 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:30.526 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211786+0000", 2024-04-25T20:10:30.526 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.6190000000000002e-06 2024-04-25T20:10:30.526 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.526 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.526 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:30.526 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213374+0000", 2024-04-25T20:10:30.526 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0015881859999999999 2024-04-25T20:10:30.526 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.526 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.526 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:30.527 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213478+0000", 2024-04-25T20:10:30.527 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000103464 2024-04-25T20:10:30.527 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.527 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.527 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:30.527 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213601+0000", 2024-04-25T20:10:30.527 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00012324300000000001 2024-04-25T20:10:30.527 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.527 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.527 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:30.527 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.214788+0000", 2024-04-25T20:10:30.527 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011868569999999999 2024-04-25T20:10:30.527 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.527 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.527 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:30.528 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227504+0000", 2024-04-25T20:10:30.528 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012715749 2024-04-25T20:10:30.528 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.528 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.528 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:30.528 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227532+0000", 2024-04-25T20:10:30.528 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.7937e-05 2024-04-25T20:10:30.528 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.528 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.528 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:30.528 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227552+0000", 2024-04-25T20:10:30.528 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0772999999999999e-05 2024-04-25T20:10:30.528 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.528 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:30.528 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.528 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.529 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.529 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:131 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:30.529 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:10:30.529 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 72.377371702000005, 2024-04-25T20:10:30.529 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0086158810000000006, 2024-04-25T20:10:30.529 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:30.529 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:30.529 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:30.529 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:30.529 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:30.529 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 131 2024-04-25T20:10:30.529 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.529 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:30.529 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.529 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:30.530 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:10:30.530 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.530 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.530 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.530 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:30.530 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:10:30.530 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.530 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.530 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.530 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:30.530 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120785+0000", 2024-04-25T20:10:30.530 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:30.530 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.531 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.531 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:30.531 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120880+0000", 2024-04-25T20:10:30.531 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.4925000000000003e-05 2024-04-25T20:10:30.531 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.531 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.531 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:30.531 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120891+0000", 2024-04-25T20:10:30.531 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0915e-05 2024-04-25T20:10:30.531 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.531 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.531 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:30.531 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120895+0000", 2024-04-25T20:10:30.531 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.8070000000000001e-06 2024-04-25T20:10:30.531 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.531 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.532 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:30.532 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121640+0000", 2024-04-25T20:10:30.532 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00074468500000000003 2024-04-25T20:10:30.532 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.532 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.532 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:30.532 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121684+0000", 2024-04-25T20:10:30.532 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4452000000000001e-05 2024-04-25T20:10:30.532 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.532 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.532 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:30.532 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121753+0000", 2024-04-25T20:10:30.532 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8508000000000002e-05 2024-04-25T20:10:30.532 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.532 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.533 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:30.533 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123618+0000", 2024-04-25T20:10:30.533 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0018655099999999999 2024-04-25T20:10:30.533 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.533 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.533 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:30.533 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129248+0000", 2024-04-25T20:10:30.533 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0056295540000000002 2024-04-25T20:10:30.533 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.533 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.533 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:30.533 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129368+0000", 2024-04-25T20:10:30.533 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000120244 2024-04-25T20:10:30.533 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.533 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.533 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:30.533 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129402+0000", 2024-04-25T20:10:30.534 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3812000000000001e-05 2024-04-25T20:10:30.534 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.534 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:30.534 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.534 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.534 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.534 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:132 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:30.534 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:30.534 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 72.377257491999998, 2024-04-25T20:10:30.534 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0085898169999999996, 2024-04-25T20:10:30.534 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:30.534 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:30.534 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:30.534 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:30.534 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:30.535 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 132 2024-04-25T20:10:30.535 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.535 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:30.535 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.535 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:30.535 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:30.535 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.535 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.535 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.535 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:30.535 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:30.535 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.535 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.535 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.536 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:30.536 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:30.536 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:30.536 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.536 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.536 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:30.536 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120902+0000", 2024-04-25T20:10:30.536 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.6129999999999998e-06 2024-04-25T20:10:30.536 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.536 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.536 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:30.536 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120906+0000", 2024-04-25T20:10:30.536 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6569999999999999e-06 2024-04-25T20:10:30.536 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.536 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.536 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:30.536 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120909+0000", 2024-04-25T20:10:30.537 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5550000000000001e-06 2024-04-25T20:10:30.537 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.537 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.537 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:30.537 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122098+0000", 2024-04-25T20:10:30.537 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011898049999999999 2024-04-25T20:10:30.537 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.537 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.537 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:30.537 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122172+0000", 2024-04-25T20:10:30.537 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3917000000000004e-05 2024-04-25T20:10:30.537 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.537 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.537 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:30.537 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122288+0000", 2024-04-25T20:10:30.537 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00011566 2024-04-25T20:10:30.538 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.538 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.538 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:30.538 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124519+0000", 2024-04-25T20:10:30.538 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0022308359999999999 2024-04-25T20:10:30.538 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.538 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.538 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:30.538 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129414+0000", 2024-04-25T20:10:30.538 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048953720000000003 2024-04-25T20:10:30.538 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.538 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.538 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:30.538 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129458+0000", 2024-04-25T20:10:30.539 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.3426000000000001e-05 2024-04-25T20:10:30.539 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.539 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.539 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:30.539 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129490+0000", 2024-04-25T20:10:30.539 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2329999999999997e-05 2024-04-25T20:10:30.539 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.539 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:30.539 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.539 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.539 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.539 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:133 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:30.539 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:30.539 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 72.377203623, 2024-04-25T20:10:30.539 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0086066270000000004, 2024-04-25T20:10:30.540 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:30.540 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:30.540 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:30.540 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:30.540 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:30.540 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 133 2024-04-25T20:10:30.540 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.540 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:30.540 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.540 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:30.540 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:30.540 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.540 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.540 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.540 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:30.540 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:30.541 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.541 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.541 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.541 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:30.541 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:30.541 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:30.541 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.541 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.541 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:30.541 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120956+0000", 2024-04-25T20:10:30.541 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5289999999999998e-06 2024-04-25T20:10:30.541 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.541 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.541 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:30.541 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120962+0000", 2024-04-25T20:10:30.542 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.9490000000000001e-06 2024-04-25T20:10:30.542 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.542 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.542 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:30.542 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120965+0000", 2024-04-25T20:10:30.542 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.1860000000000001e-06 2024-04-25T20:10:30.542 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.542 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.542 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:30.542 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122666+0000", 2024-04-25T20:10:30.542 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0017004500000000001 2024-04-25T20:10:30.542 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.542 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.542 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:30.542 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122741+0000", 2024-04-25T20:10:30.543 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.5297999999999996e-05 2024-04-25T20:10:30.543 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.543 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.543 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:30.543 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122858+0000", 2024-04-25T20:10:30.543 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00011723899999999999 2024-04-25T20:10:30.543 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.543 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.543 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:30.543 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124625+0000", 2024-04-25T20:10:30.543 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0017670100000000001 2024-04-25T20:10:30.543 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.543 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.543 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:30.543 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129497+0000", 2024-04-25T20:10:30.543 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048713580000000001 2024-04-25T20:10:30.544 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.544 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.544 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:30.544 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129532+0000", 2024-04-25T20:10:30.544 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5704999999999998e-05 2024-04-25T20:10:30.544 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.544 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.544 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:30.544 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129561+0000", 2024-04-25T20:10:30.544 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8283e-05 2024-04-25T20:10:30.544 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.544 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:30.544 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.544 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.544 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.545 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:134 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:30.545 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:30.545 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 72.377101425000006, 2024-04-25T20:10:30.545 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0085771179999999999, 2024-04-25T20:10:30.545 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:30.545 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:30.545 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:30.545 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:30.545 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:30.545 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 134 2024-04-25T20:10:30.545 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.545 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:30.545 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.545 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:30.545 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:30.546 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.546 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.546 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.546 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:30.546 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:30.546 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.546 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.546 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.546 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:30.546 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:30.546 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:30.546 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.546 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.546 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:30.546 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121157+0000", 2024-04-25T20:10:30.547 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00010104599999999999 2024-04-25T20:10:30.547 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.547 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.547 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:30.547 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121189+0000", 2024-04-25T20:10:30.547 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2589000000000003e-05 2024-04-25T20:10:30.547 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.547 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.547 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:30.547 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121195+0000", 2024-04-25T20:10:30.547 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.083e-06 2024-04-25T20:10:30.547 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.547 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.547 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:30.547 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123136+0000", 2024-04-25T20:10:30.547 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0019409780000000001 2024-04-25T20:10:30.548 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.548 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.548 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:30.548 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123181+0000", 2024-04-25T20:10:30.548 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4267999999999997e-05 2024-04-25T20:10:30.548 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.548 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.548 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:30.548 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123247+0000", 2024-04-25T20:10:30.548 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.6086000000000004e-05 2024-04-25T20:10:30.548 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.548 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.548 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:30.548 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125639+0000", 2024-04-25T20:10:30.548 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.002392 2024-04-25T20:10:30.548 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.549 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.549 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:30.549 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129567+0000", 2024-04-25T20:10:30.549 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0039282170000000003 2024-04-25T20:10:30.549 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.549 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.549 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:30.549 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129605+0000", 2024-04-25T20:10:30.549 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.7660999999999997e-05 2024-04-25T20:10:30.549 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.549 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.549 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:30.549 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129633+0000", 2024-04-25T20:10:30.549 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8649999999999998e-05 2024-04-25T20:10:30.549 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.550 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:30.550 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.550 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.550 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.550 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:135 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:30.550 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:10:30.550 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 72.376942419000002, 2024-04-25T20:10:30.550 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010898453000000001, 2024-04-25T20:10:30.550 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:30.550 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:30.550 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:30.550 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:30.550 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:30.551 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 135 2024-04-25T20:10:30.551 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.551 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:30.551 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.551 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:30.551 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:10:30.551 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.551 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.551 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.551 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:30.551 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:10:30.551 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.551 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.551 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.551 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:30.552 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121214+0000", 2024-04-25T20:10:30.552 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:30.552 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.552 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.552 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:30.552 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121218+0000", 2024-04-25T20:10:30.552 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6619999999999998e-06 2024-04-25T20:10:30.552 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.552 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.552 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:30.552 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121223+0000", 2024-04-25T20:10:30.552 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1070000000000004e-06 2024-04-25T20:10:30.552 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.552 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.552 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:30.552 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121227+0000", 2024-04-25T20:10:30.553 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2820000000000001e-06 2024-04-25T20:10:30.553 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.553 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.553 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:30.553 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123649+0000", 2024-04-25T20:10:30.553 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0024221970000000001 2024-04-25T20:10:30.553 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.553 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.553 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:30.553 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123723+0000", 2024-04-25T20:10:30.553 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.4386000000000002e-05 2024-04-25T20:10:30.553 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.553 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.553 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:30.554 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123844+0000", 2024-04-25T20:10:30.554 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000121249 2024-04-25T20:10:30.554 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.554 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.554 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:30.554 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125670+0000", 2024-04-25T20:10:30.554 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001825901 2024-04-25T20:10:30.554 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.554 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.554 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:30.554 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132043+0000", 2024-04-25T20:10:30.554 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0063724539999999996 2024-04-25T20:10:30.554 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.554 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.554 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:30.555 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132084+0000", 2024-04-25T20:10:30.555 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0963000000000001e-05 2024-04-25T20:10:30.555 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.555 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.555 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:30.557 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132114+0000", 2024-04-25T20:10:30.557 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9927e-05 2024-04-25T20:10:30.557 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.557 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:30.557 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.557 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.557 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.557 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:136 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:30.557 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:10:30.557 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 72.376926893000004, 2024-04-25T20:10:30.557 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010952513000000001, 2024-04-25T20:10:30.558 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:30.558 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:30.558 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:30.558 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:30.558 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:30.558 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 136 2024-04-25T20:10:30.558 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.558 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:30.558 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.558 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:30.558 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:10:30.558 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.558 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.558 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.559 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:30.559 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:10:30.559 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.559 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.559 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.559 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:30.559 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121230+0000", 2024-04-25T20:10:30.559 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:30.559 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.559 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.559 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:30.559 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121236+0000", 2024-04-25T20:10:30.559 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.7320000000000003e-06 2024-04-25T20:10:30.559 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.559 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.560 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:30.560 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121242+0000", 2024-04-25T20:10:30.560 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.7810000000000002e-06 2024-04-25T20:10:30.560 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.560 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.560 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:30.560 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121246+0000", 2024-04-25T20:10:30.560 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.639e-06 2024-04-25T20:10:30.560 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.560 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.560 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:30.560 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124150+0000", 2024-04-25T20:10:30.560 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.002903816 2024-04-25T20:10:30.560 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.561 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.561 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:30.561 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124195+0000", 2024-04-25T20:10:30.561 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.5042e-05 2024-04-25T20:10:30.561 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.561 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.561 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:30.561 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124266+0000", 2024-04-25T20:10:30.561 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0234999999999994e-05 2024-04-25T20:10:30.561 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.561 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.561 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:30.561 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125711+0000", 2024-04-25T20:10:30.561 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0014458800000000001 2024-04-25T20:10:30.561 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.562 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.562 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:30.562 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132120+0000", 2024-04-25T20:10:30.562 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0064084399999999996 2024-04-25T20:10:30.562 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.562 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.562 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:30.562 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132155+0000", 2024-04-25T20:10:30.562 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4938000000000003e-05 2024-04-25T20:10:30.562 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.562 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.562 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:30.562 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132183+0000", 2024-04-25T20:10:30.562 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8360999999999998e-05 2024-04-25T20:10:30.562 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.563 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:30.563 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.563 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.563 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.563 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:30.563 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:30.563 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 72.376877160999996, 2024-04-25T20:10:30.563 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.01097684, 2024-04-25T20:10:30.563 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:30.563 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:30.563 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:30.563 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:30.563 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:30.563 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 137 2024-04-25T20:10:30.563 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.564 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:30.564 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.564 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:30.564 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:30.564 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.564 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.564 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.564 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:30.564 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:30.564 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.564 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.564 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.564 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:30.564 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:30.564 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:30.564 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.565 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.565 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:30.565 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121328+0000", 2024-04-25T20:10:30.565 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.7908999999999998e-05 2024-04-25T20:10:30.565 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.565 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.565 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:30.565 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121337+0000", 2024-04-25T20:10:30.565 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.9369999999999993e-06 2024-04-25T20:10:30.565 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.565 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.565 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:30.565 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121343+0000", 2024-04-25T20:10:30.565 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.8089999999999998e-06 2024-04-25T20:10:30.566 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.566 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.566 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:30.566 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124656+0000", 2024-04-25T20:10:30.566 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0033133839999999999 2024-04-25T20:10:30.566 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.566 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.566 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:30.566 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124730+0000", 2024-04-25T20:10:30.566 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3565000000000005e-05 2024-04-25T20:10:30.566 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.566 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.566 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:30.566 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124846+0000", 2024-04-25T20:10:30.567 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000116416 2024-04-25T20:10:30.567 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.567 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.567 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:30.567 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126756+0000", 2024-04-25T20:10:30.567 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001909991 2024-04-25T20:10:30.567 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.567 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.567 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:30.567 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132190+0000", 2024-04-25T20:10:30.567 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0054343880000000001 2024-04-25T20:10:30.567 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.567 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.567 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:30.568 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132227+0000", 2024-04-25T20:10:30.568 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6875000000000002e-05 2024-04-25T20:10:30.568 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.568 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.568 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:30.568 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132257+0000", 2024-04-25T20:10:30.568 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.0046e-05 2024-04-25T20:10:30.568 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.568 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:30.568 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.568 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.568 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.568 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:30.568 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:30.569 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 72.376807202999998, 2024-04-25T20:10:30.569 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010978653999999999, 2024-04-25T20:10:30.569 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:30.569 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:30.569 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:30.569 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:30.569 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:30.569 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 138 2024-04-25T20:10:30.569 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.569 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:30.569 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.569 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:30.569 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:30.569 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.569 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.570 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.570 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:30.570 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:30.570 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.570 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.570 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.570 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:30.570 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:30.570 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:30.570 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.570 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.570 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:30.570 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121355+0000", 2024-04-25T20:10:30.570 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.9869999999999999e-06 2024-04-25T20:10:30.571 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.571 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.571 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:30.571 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121361+0000", 2024-04-25T20:10:30.571 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.4620000000000001e-06 2024-04-25T20:10:30.571 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.571 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.571 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:30.571 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121373+0000", 2024-04-25T20:10:30.571 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.2035000000000001e-05 2024-04-25T20:10:30.571 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.571 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.571 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:30.571 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125267+0000", 2024-04-25T20:10:30.572 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0038939230000000001 2024-04-25T20:10:30.572 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.572 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.572 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:30.572 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125309+0000", 2024-04-25T20:10:30.572 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1690999999999998e-05 2024-04-25T20:10:30.572 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.572 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.572 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:30.572 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125385+0000", 2024-04-25T20:10:30.572 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.6162999999999996e-05 2024-04-25T20:10:30.572 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.572 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.572 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:30.572 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126798+0000", 2024-04-25T20:10:30.573 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0014124680000000001 2024-04-25T20:10:30.573 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.573 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.573 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:30.573 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132264+0000", 2024-04-25T20:10:30.573 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0054661689999999999 2024-04-25T20:10:30.573 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.573 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.573 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:30.573 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132299+0000", 2024-04-25T20:10:30.573 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5272999999999999e-05 2024-04-25T20:10:30.573 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.573 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.573 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:30.573 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132329+0000", 2024-04-25T20:10:30.573 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.0080999999999999e-05 2024-04-25T20:10:30.574 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.574 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:30.574 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.574 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.574 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.574 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:30.574 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:10:30.574 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 72.376776628000002, 2024-04-25T20:10:30.574 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011015853000000001, 2024-04-25T20:10:30.574 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:30.574 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:30.574 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:30.574 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:30.574 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:30.574 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 139 2024-04-25T20:10:30.574 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.575 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:30.575 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.575 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:30.575 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:10:30.575 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.575 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.575 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.575 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:30.575 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:10:30.575 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.575 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.575 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.575 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:30.575 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121380+0000", 2024-04-25T20:10:30.575 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:30.575 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.576 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.576 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:30.576 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121385+0000", 2024-04-25T20:10:30.576 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.092e-06 2024-04-25T20:10:30.576 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.576 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.576 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:30.576 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121390+0000", 2024-04-25T20:10:30.576 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.5990000000000001e-06 2024-04-25T20:10:30.576 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.576 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.576 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:30.576 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121397+0000", 2024-04-25T20:10:30.576 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.6279999999999999e-06 2024-04-25T20:10:30.576 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.576 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.577 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:30.577 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125737+0000", 2024-04-25T20:10:30.577 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0043406010000000004 2024-04-25T20:10:30.577 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.577 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.577 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:30.577 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125781+0000", 2024-04-25T20:10:30.577 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.3958999999999997e-05 2024-04-25T20:10:30.577 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.577 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.577 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:30.577 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125849+0000", 2024-04-25T20:10:30.577 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8071999999999995e-05 2024-04-25T20:10:30.577 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.577 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.578 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:30.578 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.130908+0000", 2024-04-25T20:10:30.578 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0050586509999999999 2024-04-25T20:10:30.578 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.578 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.578 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:30.578 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132334+0000", 2024-04-25T20:10:30.578 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00142621 2024-04-25T20:10:30.578 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.578 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.578 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:30.578 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132369+0000", 2024-04-25T20:10:30.578 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4452000000000002e-05 2024-04-25T20:10:30.578 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.579 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.579 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:30.579 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132397+0000", 2024-04-25T20:10:30.579 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8065e-05 2024-04-25T20:10:30.579 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.579 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:30.579 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.579 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.579 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.579 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:30.579 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:30.579 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 72.376755583999994, 2024-04-25T20:10:30.579 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011069443, 2024-04-25T20:10:30.579 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:30.579 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:30.580 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:30.580 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:30.580 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:30.580 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 140 2024-04-25T20:10:30.580 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.580 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:30.580 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.580 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:30.580 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:30.580 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.580 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.580 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.580 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:30.580 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:30.580 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.580 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.581 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.581 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:30.581 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:30.581 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:30.581 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.581 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.581 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:30.581 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121456+0000", 2024-04-25T20:10:30.581 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.4469000000000003e-05 2024-04-25T20:10:30.581 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.581 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.581 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:30.581 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121462+0000", 2024-04-25T20:10:30.581 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.1890000000000004e-06 2024-04-25T20:10:30.581 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.582 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.582 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:30.582 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121466+0000", 2024-04-25T20:10:30.582 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3909999999999998e-06 2024-04-25T20:10:30.582 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.582 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.582 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:30.582 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126071+0000", 2024-04-25T20:10:30.582 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0046048979999999996 2024-04-25T20:10:30.582 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.582 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.582 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:30.582 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126141+0000", 2024-04-25T20:10:30.582 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0621e-05 2024-04-25T20:10:30.582 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.583 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.583 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:30.583 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126260+0000", 2024-04-25T20:10:30.583 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000118621 2024-04-25T20:10:30.583 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.583 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.583 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:30.583 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.131270+0000", 2024-04-25T20:10:30.583 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0050096949999999998 2024-04-25T20:10:30.583 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.583 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.583 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:30.583 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132403+0000", 2024-04-25T20:10:30.583 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011332779999999999 2024-04-25T20:10:30.584 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.584 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.584 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:30.584 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132443+0000", 2024-04-25T20:10:30.584 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0175000000000001e-05 2024-04-25T20:10:30.584 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.584 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.584 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:30.584 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132471+0000", 2024-04-25T20:10:30.584 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8399e-05 2024-04-25T20:10:30.584 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.584 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:30.584 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.584 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.584 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.585 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:30.585 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:10:30.585 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 72.376686798999998, 2024-04-25T20:10:30.585 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011071002, 2024-04-25T20:10:30.585 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:30.585 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:30.585 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:30.585 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:30.585 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:30.585 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 141 2024-04-25T20:10:30.585 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.585 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:30.585 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.585 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:30.586 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:10:30.586 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.586 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.586 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.586 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:30.586 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:10:30.586 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.586 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.586 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.586 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:30.586 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121470+0000", 2024-04-25T20:10:30.586 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:30.586 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.586 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.586 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:30.587 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121473+0000", 2024-04-25T20:10:30.587 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.01e-06 2024-04-25T20:10:30.587 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.587 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.587 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:30.587 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121478+0000", 2024-04-25T20:10:30.587 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.515e-06 2024-04-25T20:10:30.587 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.587 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.587 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:30.587 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121480+0000", 2024-04-25T20:10:30.587 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5749999999999999e-06 2024-04-25T20:10:30.587 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.587 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.587 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:30.587 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126813+0000", 2024-04-25T20:10:30.588 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0053320399999999997 2024-04-25T20:10:30.588 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.588 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.588 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:30.588 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126854+0000", 2024-04-25T20:10:30.588 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1582999999999998e-05 2024-04-25T20:10:30.588 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.588 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.588 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:30.588 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126923+0000", 2024-04-25T20:10:30.588 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8544e-05 2024-04-25T20:10:30.588 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.588 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.588 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:30.588 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.131573+0000", 2024-04-25T20:10:30.589 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0046504659999999998 2024-04-25T20:10:30.589 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.589 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.589 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:30.589 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132478+0000", 2024-04-25T20:10:30.589 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00090442400000000003 2024-04-25T20:10:30.589 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.589 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.589 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:30.589 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132513+0000", 2024-04-25T20:10:30.589 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5204999999999999e-05 2024-04-25T20:10:30.589 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.589 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.589 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:30.589 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132542+0000", 2024-04-25T20:10:30.589 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.904e-05 2024-04-25T20:10:30.590 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.590 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:30.590 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.590 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.590 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.590 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:156 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:30.590 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:30.590 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 71.878862503999997, 2024-04-25T20:10:30.590 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090072280000000008, 2024-04-25T20:10:30.590 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:30.590 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:30.590 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:30.590 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:30.590 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:30.590 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 156 2024-04-25T20:10:30.591 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.591 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:30.591 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.591 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:30.591 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:30.591 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.591 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.591 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.591 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:30.591 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:30.591 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.591 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.591 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.591 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:30.592 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:30.592 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:30.592 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.592 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.592 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:30.592 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619297+0000", 2024-04-25T20:10:30.592 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2979999999999999e-06 2024-04-25T20:10:30.592 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.592 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.592 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:30.592 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619300+0000", 2024-04-25T20:10:30.592 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.371e-06 2024-04-25T20:10:30.592 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.592 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.592 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:30.592 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619304+0000", 2024-04-25T20:10:30.593 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.304e-06 2024-04-25T20:10:30.593 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.593 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.593 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:30.593 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.622910+0000", 2024-04-25T20:10:30.593 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0036065049999999999 2024-04-25T20:10:30.593 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.593 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.593 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:30.593 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.622990+0000", 2024-04-25T20:10:30.593 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.9722999999999995e-05 2024-04-25T20:10:30.593 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.593 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.593 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:30.593 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623128+0000", 2024-04-25T20:10:30.593 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00013818399999999999 2024-04-25T20:10:30.594 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.594 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.594 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:30.594 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625155+0000", 2024-04-25T20:10:30.594 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0020267979999999998 2024-04-25T20:10:30.594 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.594 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.594 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:30.594 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628225+0000", 2024-04-25T20:10:30.594 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0030705680000000001 2024-04-25T20:10:30.594 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.594 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.594 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:30.594 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628268+0000", 2024-04-25T20:10:30.595 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.2404000000000003e-05 2024-04-25T20:10:30.595 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.595 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.595 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:30.595 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628302+0000", 2024-04-25T20:10:30.595 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4439e-05 2024-04-25T20:10:30.595 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.595 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:30.595 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.595 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.595 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.595 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:157 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:30.595 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:30.595 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 71.878850404999994, 2024-04-25T20:10:30.596 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090534889999999996, 2024-04-25T20:10:30.596 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:30.596 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:30.596 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:30.596 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:30.596 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:30.596 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 157 2024-04-25T20:10:30.596 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.596 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:30.596 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.596 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:30.596 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:30.596 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.596 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.596 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.597 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:30.597 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:30.597 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.597 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.597 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.597 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:30.597 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:30.597 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:30.597 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.597 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.597 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:30.597 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619309+0000", 2024-04-25T20:10:30.597 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.841e-06 2024-04-25T20:10:30.597 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.597 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.598 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:30.598 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619312+0000", 2024-04-25T20:10:30.598 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.084e-06 2024-04-25T20:10:30.598 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.598 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.598 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:30.598 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619314+0000", 2024-04-25T20:10:30.598 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.3860000000000001e-06 2024-04-25T20:10:30.598 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.598 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.598 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:30.598 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623718+0000", 2024-04-25T20:10:30.598 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0044038050000000002 2024-04-25T20:10:30.598 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.599 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.599 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:30.599 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623791+0000", 2024-04-25T20:10:30.599 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3181000000000003e-05 2024-04-25T20:10:30.599 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.599 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.599 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:30.599 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623922+0000", 2024-04-25T20:10:30.599 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00013117000000000001 2024-04-25T20:10:30.599 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.599 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.599 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:30.599 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625187+0000", 2024-04-25T20:10:30.599 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001264961 2024-04-25T20:10:30.599 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.600 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.600 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:30.600 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628308+0000", 2024-04-25T20:10:30.600 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.003120739 2024-04-25T20:10:30.600 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.600 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.600 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:30.600 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628331+0000", 2024-04-25T20:10:30.600 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2802999999999998e-05 2024-04-25T20:10:30.600 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.600 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.600 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:30.600 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628361+0000", 2024-04-25T20:10:30.600 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9760999999999998e-05 2024-04-25T20:10:30.600 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.601 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:30.601 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.601 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.601 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.601 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:158 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:30.601 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:10:30.601 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 71.878839958, 2024-04-25T20:10:30.601 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0091076409999999997, 2024-04-25T20:10:30.601 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:30.601 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:30.601 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:30.601 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:30.601 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:30.601 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 158 2024-04-25T20:10:30.602 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.602 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:30.602 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.602 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:30.602 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:10:30.602 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.602 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.602 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.602 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:30.602 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:10:30.602 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.602 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.602 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.602 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:30.602 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619317+0000", 2024-04-25T20:10:30.603 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967296 2024-04-25T20:10:30.603 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.603 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.603 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:30.603 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619364+0000", 2024-04-25T20:10:30.603 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.6572000000000001e-05 2024-04-25T20:10:30.603 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.603 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.603 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:30.603 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619371+0000", 2024-04-25T20:10:30.603 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3860000000000001e-06 2024-04-25T20:10:30.603 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.603 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.603 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:30.603 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619382+0000", 2024-04-25T20:10:30.604 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0173999999999999e-05 2024-04-25T20:10:30.604 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.604 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.604 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:30.604 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624216+0000", 2024-04-25T20:10:30.604 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048340350000000004 2024-04-25T20:10:30.604 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.604 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.604 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:30.604 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624260+0000", 2024-04-25T20:10:30.604 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4558999999999998e-05 2024-04-25T20:10:30.604 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.605 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.605 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:30.605 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624333+0000", 2024-04-25T20:10:30.605 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2744999999999996e-05 2024-04-25T20:10:30.605 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.605 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.605 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:30.605 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625465+0000", 2024-04-25T20:10:30.605 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001132454 2024-04-25T20:10:30.605 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.605 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.605 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:30.605 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628365+0000", 2024-04-25T20:10:30.605 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0028994950000000002 2024-04-25T20:10:30.605 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.606 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.606 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:30.606 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628386+0000", 2024-04-25T20:10:30.606 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.1367000000000001e-05 2024-04-25T20:10:30.606 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.606 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.606 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:30.606 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628425+0000", 2024-04-25T20:10:30.606 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.9087e-05 2024-04-25T20:10:30.606 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.606 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:30.606 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.606 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.606 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.606 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:159 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:30.607 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:30.607 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 71.878771305000001, 2024-04-25T20:10:30.607 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090948999999999995, 2024-04-25T20:10:30.607 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:30.607 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:30.607 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:30.607 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:30.607 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:30.607 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 159 2024-04-25T20:10:30.607 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.607 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:30.607 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.607 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:30.607 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:30.607 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.607 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.608 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.608 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:30.608 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:30.608 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:30.608 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.608 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.608 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:30.608 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:30.608 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:30.608 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.608 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.608 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:30.608 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619389+0000", 2024-04-25T20:10:30.608 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.734e-06 2024-04-25T20:10:30.608 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.608 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.608 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:30.609 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619392+0000", 2024-04-25T20:10:30.609 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6219999999999998e-06 2024-04-25T20:10:30.609 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.609 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.609 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:30.609 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619403+0000", 2024-04-25T20:10:30.609 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0987e-05 2024-04-25T20:10:30.609 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.609 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.609 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:30.609 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624667+0000", 2024-04-25T20:10:30.609 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0052633970000000004 2024-04-25T20:10:30.609 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.609 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.609 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:30.609 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624741+0000", 2024-04-25T20:10:30.610 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.4583000000000001e-05 2024-04-25T20:10:30.610 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.610 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.610 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:30.610 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624857+0000", 2024-04-25T20:10:30.610 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000116118 2024-04-25T20:10:30.610 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.610 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.610 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:30.610 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.626013+0000", 2024-04-25T20:10:30.610 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001155473 2024-04-25T20:10:30.610 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.610 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.610 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:30.611 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628432+0000", 2024-04-25T20:10:30.611 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0024189379999999998 2024-04-25T20:10:30.611 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.611 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.611 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:30.611 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628454+0000", 2024-04-25T20:10:30.611 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2059999999999999e-05 2024-04-25T20:10:30.611 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:30.611 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:30.611 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:30.611 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628481+0000", 2024-04-25T20:10:30.611 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.7300999999999998e-05 2024-04-25T20:10:30.611 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.611 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:30.611 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.612 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:30.612 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:30.612 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:30.612 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:30.499+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:30.613 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T20:10:30.614 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:30.614+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:30.715 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:30.715+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:30.722 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:30.723 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:30.723 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:30.723 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:30.730 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is set 2024-04-25T20:10:30.735 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T20:10:30.830 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T20:10:30.833 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:30.833+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:30.917 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:30.917+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:30.956 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:30.956 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:30.956 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:30.957 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:30.957 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:30.970 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T20:10:31.010 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:10:31.010 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:10:31.032 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:10:31.032 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:10:31.033 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:31.032+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:31.065 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:10:31.065 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:10:31.097 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:10:31.097 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:10:31.119 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:31.119+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:31.129 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:10:31.129 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:10:31.161 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:10:31.162 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:10:31.180 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:31.180 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:31.192 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T20:10:31.197 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:10:31.197 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:10:31.220 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:31.220+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:31.228 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:10:31.228 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:10:31.321 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:31.321+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:31.361 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:31.361 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:31.373 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T20:10:31.457 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:31.457+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:31.522 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:31.522+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:31.587 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:31.587 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:31.600 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T20:10:31.623 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:31.622+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:31.706 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7] out_osds: [3, 2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-25T20:10:31.706 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T20:10:31.706 INFO:tasks.thrashosds.thrasher:Reweighting osd 7 to 0.945918880426422 2024-04-25T20:10:31.707 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 0.945918880426422 2024-04-25T20:10:31.724 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:31.723+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:31.775 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:31.775 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:31.784 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T20:10:31.825 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:31.824+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:31.895 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:31.895 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:31.904 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T20:10:31.926 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:31.925+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:32.012 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:32.012 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:32.021 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T20:10:32.064 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:32.064+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:32.127 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:32.127+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:32.129 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:32.129 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:32.139 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T20:10:32.228 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:32.228+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:32.287 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:32.287 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:32.297 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T20:10:32.340 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:32.340+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:32.406 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:32.406 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:32.415 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T20:10:32.430 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:32.430+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:32.531 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:32.531+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:32.624 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:32.624 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:32.632 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:32.632+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:32.636 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T20:10:32.734 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:32.733+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:32.750 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:32.750 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:32.750 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-25T20:10:32.759 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.7 to 0.945919 (f227) 2024-04-25T20:10:32.760 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T20:10:32.834 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:32.834+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:32.862 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:32.862 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:32.871 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T20:10:32.935 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:32.935+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:32.972 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:32.972 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:32.981 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T20:10:33.066 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:33.065+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:33.123 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:33.123 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:33.132 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T20:10:33.138 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:33.137+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:33.238 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:33.238+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:33.242 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T20:10:33.274 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:33.275 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:33.283 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T20:10:33.339 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:33.339+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:33.385 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:33.385 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:33.394 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T20:10:33.400 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:10:33.400 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:10:33.419 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:10:33.419 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:10:33.437 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:10:33.437 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:10:33.440 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:33.440+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:33.454 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:10:33.454 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:10:33.471 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:10:33.471 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:10:33.487 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:10:33.488 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:10:33.497 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:33.497 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:33.504 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:10:33.504 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:10:33.506 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T20:10:33.521 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:10:33.521 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:10:33.551 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:33.550+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:33.607 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:33.607 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:33.607 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:33.607 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:33.607 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:33.616 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T20:10:33.658 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:33.657+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:33.742 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:33.741+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:33.758 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:33.759 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:33.759 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:33.759 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:10:33.759 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:33.768 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T20:10:33.810 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is unset 2024-04-25T20:10:33.869 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:33.869+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:33.870 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:33.870 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:33.870 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:10:33.870 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:33.879 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T20:10:33.944 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:33.943+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:33.981 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:33.981 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:33.981 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:33.981 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:33.981 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:33.990 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T20:10:34.075 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:34.074+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:34.134 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:34.134 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:34.134 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:34.134 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [ 2024-04-25T20:10:34.135 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.135 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:34.135 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:10:34.135 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 77.350599379000002, 2024-04-25T20:10:34.135 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.34154860100000001, 2024-04-25T20:10:34.135 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:34.135 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:34.135 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:34.135 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:34.135 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:34.135 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 1 2024-04-25T20:10:34.135 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.136 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:34.136 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.136 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:34.136 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:10:34.136 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.136 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.136 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.136 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:34.136 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:10:34.136 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.136 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.136 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.136 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:34.136 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782261+0000", 2024-04-25T20:10:34.137 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999986 2024-04-25T20:10:34.137 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.137 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.137 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:34.137 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782268+0000", 2024-04-25T20:10:34.137 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.8140000000000001e-06 2024-04-25T20:10:34.137 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.137 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.137 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:34.137 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782279+0000", 2024-04-25T20:10:34.137 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0665999999999999e-05 2024-04-25T20:10:34.137 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.137 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.137 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:34.138 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782290+0000", 2024-04-25T20:10:34.138 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.1343e-05 2024-04-25T20:10:34.138 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.138 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.138 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:34.138 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782340+0000", 2024-04-25T20:10:34.138 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.9311999999999998e-05 2024-04-25T20:10:34.138 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.138 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.138 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for peered", 2024-04-25T20:10:34.138 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782348+0000", 2024-04-25T20:10:34.138 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.0690000000000002e-06 2024-04-25T20:10:34.139 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.139 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.139 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:34.139 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120520+0000", 2024-04-25T20:10:34.139 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.33817188100000001 2024-04-25T20:10:34.139 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.139 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.139 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:34.139 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120656+0000", 2024-04-25T20:10:34.139 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000135897 2024-04-25T20:10:34.139 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.139 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.139 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:34.139 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120802+0000", 2024-04-25T20:10:34.140 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00014691199999999999 2024-04-25T20:10:34.140 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.140 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.140 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:34.140 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.122368+0000", 2024-04-25T20:10:34.140 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0015651040000000001 2024-04-25T20:10:34.140 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.140 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.140 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:34.140 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123729+0000", 2024-04-25T20:10:34.140 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0013610499999999999 2024-04-25T20:10:34.140 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.140 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.140 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:34.141 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123769+0000", 2024-04-25T20:10:34.141 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0719000000000001e-05 2024-04-25T20:10:34.141 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.141 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.141 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:34.141 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123811+0000", 2024-04-25T20:10:34.141 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1267999999999999e-05 2024-04-25T20:10:34.141 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.141 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:34.141 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.141 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.141 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.141 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:67 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:34.141 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:10:34.141 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 76.921180492000005, 2024-04-25T20:10:34.141 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.006947651, 2024-04-25T20:10:34.141 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:34.142 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:34.142 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:34.142 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:34.142 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:34.142 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 67 2024-04-25T20:10:34.142 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.142 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:34.142 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.142 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:34.142 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:10:34.142 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.142 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.142 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.143 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:34.143 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:10:34.143 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.143 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.143 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.143 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:34.143 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211680+0000", 2024-04-25T20:10:34.143 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:34.143 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.143 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.143 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:34.143 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211684+0000", 2024-04-25T20:10:34.143 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4249999999999998e-06 2024-04-25T20:10:34.143 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.144 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.144 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:34.144 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211690+0000", 2024-04-25T20:10:34.144 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.7360000000000004e-06 2024-04-25T20:10:34.144 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.144 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.144 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:34.144 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211694+0000", 2024-04-25T20:10:34.144 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4860000000000002e-06 2024-04-25T20:10:34.144 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.144 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.144 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:34.144 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211727+0000", 2024-04-25T20:10:34.144 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3445999999999998e-05 2024-04-25T20:10:34.145 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.145 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.145 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:34.145 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211778+0000", 2024-04-25T20:10:34.145 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1050000000000001e-05 2024-04-25T20:10:34.145 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.145 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.145 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:34.145 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211851+0000", 2024-04-25T20:10:34.145 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2914e-05 2024-04-25T20:10:34.145 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.145 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.145 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:34.145 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213824+0000", 2024-04-25T20:10:34.146 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0019726520000000001 2024-04-25T20:10:34.146 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.146 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.146 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:34.146 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218579+0000", 2024-04-25T20:10:34.146 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0047553159999999999 2024-04-25T20:10:34.146 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.146 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.146 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:34.146 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218607+0000", 2024-04-25T20:10:34.146 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8144999999999999e-05 2024-04-25T20:10:34.146 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.146 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.146 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:34.146 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218629+0000", 2024-04-25T20:10:34.147 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.1044e-05 2024-04-25T20:10:34.147 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.147 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:34.147 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.147 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.147 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.147 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:68 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:34.147 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:34.147 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 76.921162151000004, 2024-04-25T20:10:34.147 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012255596000000001, 2024-04-25T20:10:34.147 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:34.147 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:34.147 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:34.147 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:34.147 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:34.147 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 68 2024-04-25T20:10:34.148 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.148 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:34.148 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.148 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:34.148 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:34.148 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.148 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.148 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.148 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:34.148 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:34.148 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.148 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.148 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.148 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:34.148 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:34.149 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:34.149 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.149 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.149 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:34.149 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211701+0000", 2024-04-25T20:10:34.149 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2029999999999999e-06 2024-04-25T20:10:34.149 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.149 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.149 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:34.149 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211705+0000", 2024-04-25T20:10:34.149 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.2320000000000003e-06 2024-04-25T20:10:34.149 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.149 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.149 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:34.149 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211708+0000", 2024-04-25T20:10:34.150 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.6290000000000002e-06 2024-04-25T20:10:34.150 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.150 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.150 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:34.150 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212136+0000", 2024-04-25T20:10:34.150 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00042764099999999998 2024-04-25T20:10:34.150 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.150 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.150 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:34.150 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212212+0000", 2024-04-25T20:10:34.150 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.6315000000000004e-05 2024-04-25T20:10:34.150 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.150 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.150 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:34.150 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212306+0000", 2024-04-25T20:10:34.151 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.3556999999999999e-05 2024-04-25T20:10:34.151 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.151 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.151 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:34.151 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213829+0000", 2024-04-25T20:10:34.151 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001523256 2024-04-25T20:10:34.151 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.151 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.151 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:34.151 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223904+0000", 2024-04-25T20:10:34.151 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010075387 2024-04-25T20:10:34.151 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.151 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.151 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:34.151 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223934+0000", 2024-04-25T20:10:34.151 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9988000000000002e-05 2024-04-25T20:10:34.151 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.152 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.152 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:34.152 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223955+0000", 2024-04-25T20:10:34.152 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0630999999999999e-05 2024-04-25T20:10:34.152 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.152 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:34.152 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.152 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.152 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.152 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:69 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:34.152 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:34.152 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 76.921140416, 2024-04-25T20:10:34.152 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012316863000000001, 2024-04-25T20:10:34.152 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:34.153 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:34.153 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:34.153 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:34.153 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:34.153 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 69 2024-04-25T20:10:34.153 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.153 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:34.153 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.153 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:34.153 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:34.153 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.153 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.153 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.153 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:34.153 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:34.154 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.154 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.154 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.154 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:34.154 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:34.154 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:34.154 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.154 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.154 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:34.154 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211762+0000", 2024-04-25T20:10:34.154 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1703999999999999e-05 2024-04-25T20:10:34.154 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.154 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.154 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:34.154 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211768+0000", 2024-04-25T20:10:34.155 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.3430000000000003e-06 2024-04-25T20:10:34.155 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.155 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.155 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:34.155 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211771+0000", 2024-04-25T20:10:34.155 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.18e-06 2024-04-25T20:10:34.155 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.155 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.155 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:34.155 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212652+0000", 2024-04-25T20:10:34.155 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00088095800000000004 2024-04-25T20:10:34.155 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.155 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.155 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:34.155 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212703+0000", 2024-04-25T20:10:34.156 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.0655999999999997e-05 2024-04-25T20:10:34.156 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.156 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.156 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:34.156 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212781+0000", 2024-04-25T20:10:34.156 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.8852999999999999e-05 2024-04-25T20:10:34.156 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.156 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.156 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:34.156 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.214131+0000", 2024-04-25T20:10:34.156 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0013496039999999999 2024-04-25T20:10:34.156 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.156 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.156 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:34.156 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223986+0000", 2024-04-25T20:10:34.157 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.009854804 2024-04-25T20:10:34.157 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.157 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.157 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:34.157 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.224017+0000", 2024-04-25T20:10:34.157 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.1068999999999997e-05 2024-04-25T20:10:34.157 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.157 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.157 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:34.157 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.224038+0000", 2024-04-25T20:10:34.157 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0965e-05 2024-04-25T20:10:34.157 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.157 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:34.157 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.158 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.158 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.158 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:70 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:34.158 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:34.158 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 76.921085335000001, 2024-04-25T20:10:34.158 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.015776348999999999, 2024-04-25T20:10:34.158 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:34.158 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:34.158 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:34.158 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:34.158 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:34.158 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 70 2024-04-25T20:10:34.158 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.158 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:34.159 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.159 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:34.159 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:34.159 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.159 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.159 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.159 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:34.159 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:34.159 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.159 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.159 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.159 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:34.159 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:34.159 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:34.159 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.159 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.160 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:34.160 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211778+0000", 2024-04-25T20:10:34.160 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5330000000000001e-06 2024-04-25T20:10:34.160 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.160 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.160 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:34.160 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211782+0000", 2024-04-25T20:10:34.160 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4079999999999998e-06 2024-04-25T20:10:34.160 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.160 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.160 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:34.160 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211786+0000", 2024-04-25T20:10:34.160 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.6190000000000002e-06 2024-04-25T20:10:34.161 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.161 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.161 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:34.161 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213374+0000", 2024-04-25T20:10:34.161 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0015881859999999999 2024-04-25T20:10:34.161 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.161 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.161 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:34.161 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213478+0000", 2024-04-25T20:10:34.161 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000103464 2024-04-25T20:10:34.161 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.161 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.161 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:34.161 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213601+0000", 2024-04-25T20:10:34.161 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00012324300000000001 2024-04-25T20:10:34.162 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.162 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.162 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:34.162 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.214788+0000", 2024-04-25T20:10:34.162 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011868569999999999 2024-04-25T20:10:34.162 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.162 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.162 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:34.162 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227504+0000", 2024-04-25T20:10:34.162 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012715749 2024-04-25T20:10:34.162 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.162 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.162 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:34.162 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227532+0000", 2024-04-25T20:10:34.163 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.7937e-05 2024-04-25T20:10:34.163 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.163 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.163 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:34.163 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227552+0000", 2024-04-25T20:10:34.163 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0772999999999999e-05 2024-04-25T20:10:34.163 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.163 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:34.163 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.163 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.163 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.163 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:131 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:34.163 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:10:34.163 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 76.012075490000001, 2024-04-25T20:10:34.163 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0086158810000000006, 2024-04-25T20:10:34.163 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:34.164 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:34.164 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:34.164 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:34.164 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:34.164 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 131 2024-04-25T20:10:34.164 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.164 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:34.164 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.164 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:34.164 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:10:34.164 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.164 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.164 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.164 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:34.165 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:10:34.165 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.165 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.165 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.165 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:34.165 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120785+0000", 2024-04-25T20:10:34.165 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:34.165 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.165 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.165 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:34.165 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120880+0000", 2024-04-25T20:10:34.165 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.4925000000000003e-05 2024-04-25T20:10:34.165 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.165 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.165 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:34.166 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120891+0000", 2024-04-25T20:10:34.166 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0915e-05 2024-04-25T20:10:34.166 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.166 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.166 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:34.166 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120895+0000", 2024-04-25T20:10:34.166 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.8070000000000001e-06 2024-04-25T20:10:34.166 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.166 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.166 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:34.166 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121640+0000", 2024-04-25T20:10:34.166 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00074468500000000003 2024-04-25T20:10:34.166 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.166 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.166 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:34.166 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121684+0000", 2024-04-25T20:10:34.167 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4452000000000001e-05 2024-04-25T20:10:34.167 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.167 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.167 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:34.167 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121753+0000", 2024-04-25T20:10:34.167 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8508000000000002e-05 2024-04-25T20:10:34.167 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.167 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.167 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:34.167 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123618+0000", 2024-04-25T20:10:34.167 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0018655099999999999 2024-04-25T20:10:34.167 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.167 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.167 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:34.167 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129248+0000", 2024-04-25T20:10:34.168 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0056295540000000002 2024-04-25T20:10:34.168 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.168 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.168 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:34.168 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129368+0000", 2024-04-25T20:10:34.168 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000120244 2024-04-25T20:10:34.168 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.168 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.168 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:34.168 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129402+0000", 2024-04-25T20:10:34.168 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3812000000000001e-05 2024-04-25T20:10:34.168 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.168 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:34.168 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.168 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.168 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.169 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:132 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:34.169 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:34.169 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 76.011961279999994, 2024-04-25T20:10:34.169 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0085898169999999996, 2024-04-25T20:10:34.169 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:34.169 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:34.169 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:34.169 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:34.169 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:34.169 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 132 2024-04-25T20:10:34.169 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.169 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:34.169 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.169 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:34.169 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:34.169 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.170 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.170 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.170 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:34.170 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:34.170 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.170 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.170 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.170 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:34.170 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:34.170 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:34.170 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.170 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.170 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:34.170 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120902+0000", 2024-04-25T20:10:34.171 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.6129999999999998e-06 2024-04-25T20:10:34.171 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.171 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.171 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:34.171 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120906+0000", 2024-04-25T20:10:34.171 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6569999999999999e-06 2024-04-25T20:10:34.171 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.171 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.171 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:34.171 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120909+0000", 2024-04-25T20:10:34.171 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5550000000000001e-06 2024-04-25T20:10:34.171 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.171 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.171 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:34.171 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122098+0000", 2024-04-25T20:10:34.171 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011898049999999999 2024-04-25T20:10:34.172 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.172 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.172 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:34.172 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122172+0000", 2024-04-25T20:10:34.172 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3917000000000004e-05 2024-04-25T20:10:34.172 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.172 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.172 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:34.172 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122288+0000", 2024-04-25T20:10:34.172 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00011566 2024-04-25T20:10:34.172 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.172 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.172 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:34.172 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124519+0000", 2024-04-25T20:10:34.172 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0022308359999999999 2024-04-25T20:10:34.173 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.173 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.173 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:34.173 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129414+0000", 2024-04-25T20:10:34.173 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048953720000000003 2024-04-25T20:10:34.173 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.173 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.173 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:34.173 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129458+0000", 2024-04-25T20:10:34.173 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.3426000000000001e-05 2024-04-25T20:10:34.173 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.173 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.173 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:34.173 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129490+0000", 2024-04-25T20:10:34.173 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2329999999999997e-05 2024-04-25T20:10:34.174 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.174 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:34.174 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.174 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.174 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.174 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:133 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:34.174 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:34.174 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 76.011907410999996, 2024-04-25T20:10:34.174 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0086066270000000004, 2024-04-25T20:10:34.174 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:34.174 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:34.174 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:34.174 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:34.174 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:34.175 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 133 2024-04-25T20:10:34.175 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.175 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:34.175 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.175 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:34.175 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:34.175 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.175 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.175 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.175 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:34.175 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:34.175 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.175 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.175 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.175 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:34.176 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:34.176 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:34.176 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.176 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.176 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:34.176 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120956+0000", 2024-04-25T20:10:34.176 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5289999999999998e-06 2024-04-25T20:10:34.176 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.176 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.176 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:34.176 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120962+0000", 2024-04-25T20:10:34.176 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.9490000000000001e-06 2024-04-25T20:10:34.176 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.176 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.176 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:34.176 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120965+0000", 2024-04-25T20:10:34.177 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.1860000000000001e-06 2024-04-25T20:10:34.177 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.177 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.177 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:34.177 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122666+0000", 2024-04-25T20:10:34.177 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0017004500000000001 2024-04-25T20:10:34.177 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.177 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.177 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:34.177 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122741+0000", 2024-04-25T20:10:34.177 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.5297999999999996e-05 2024-04-25T20:10:34.177 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.177 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.177 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:34.177 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122858+0000", 2024-04-25T20:10:34.178 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00011723899999999999 2024-04-25T20:10:34.178 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.178 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.178 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:34.178 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124625+0000", 2024-04-25T20:10:34.178 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0017670100000000001 2024-04-25T20:10:34.178 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.178 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.178 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:34.178 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129497+0000", 2024-04-25T20:10:34.178 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048713580000000001 2024-04-25T20:10:34.178 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.178 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.178 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:34.178 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129532+0000", 2024-04-25T20:10:34.179 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5704999999999998e-05 2024-04-25T20:10:34.179 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.179 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.179 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:34.179 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129561+0000", 2024-04-25T20:10:34.179 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8283e-05 2024-04-25T20:10:34.179 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.179 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:34.179 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.179 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.179 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.179 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:134 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:34.179 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:34.179 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 76.011805213000002, 2024-04-25T20:10:34.179 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0085771179999999999, 2024-04-25T20:10:34.180 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:34.180 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:34.180 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:34.180 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:34.180 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:34.180 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 134 2024-04-25T20:10:34.180 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.180 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:34.180 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.180 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:34.180 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:34.180 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.180 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.180 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.180 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:34.180 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:34.181 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.181 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.181 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.181 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:34.181 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:34.181 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:34.181 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.181 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.181 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:34.181 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121157+0000", 2024-04-25T20:10:34.182 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00010104599999999999 2024-04-25T20:10:34.182 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.182 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.182 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:34.182 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121189+0000", 2024-04-25T20:10:34.182 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2589000000000003e-05 2024-04-25T20:10:34.182 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.182 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.182 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:34.182 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121195+0000", 2024-04-25T20:10:34.182 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.083e-06 2024-04-25T20:10:34.182 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.182 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.182 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:34.183 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123136+0000", 2024-04-25T20:10:34.183 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0019409780000000001 2024-04-25T20:10:34.183 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.183 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.183 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:34.183 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123181+0000", 2024-04-25T20:10:34.183 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4267999999999997e-05 2024-04-25T20:10:34.183 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.183 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.183 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:34.183 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123247+0000", 2024-04-25T20:10:34.183 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.6086000000000004e-05 2024-04-25T20:10:34.183 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.183 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.183 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:34.183 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125639+0000", 2024-04-25T20:10:34.184 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.002392 2024-04-25T20:10:34.184 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.184 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.184 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:34.184 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129567+0000", 2024-04-25T20:10:34.184 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0039282170000000003 2024-04-25T20:10:34.184 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.184 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.184 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:34.184 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129605+0000", 2024-04-25T20:10:34.184 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.7660999999999997e-05 2024-04-25T20:10:34.184 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.184 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.184 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:34.184 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129633+0000", 2024-04-25T20:10:34.184 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8649999999999998e-05 2024-04-25T20:10:34.184 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.184 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:34.185 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.185 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.185 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.185 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:135 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:34.185 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:10:34.185 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 76.011646206999998, 2024-04-25T20:10:34.185 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010898453000000001, 2024-04-25T20:10:34.185 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:34.185 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:34.185 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:34.185 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:34.185 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:34.185 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 135 2024-04-25T20:10:34.185 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.185 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:34.185 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.186 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:34.186 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:10:34.186 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.186 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.186 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.186 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:34.186 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:10:34.186 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.186 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.186 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.186 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:34.186 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121214+0000", 2024-04-25T20:10:34.186 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:34.186 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.186 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.186 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:34.187 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121218+0000", 2024-04-25T20:10:34.187 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6619999999999998e-06 2024-04-25T20:10:34.187 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.187 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.187 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:34.187 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121223+0000", 2024-04-25T20:10:34.187 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1070000000000004e-06 2024-04-25T20:10:34.187 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.187 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.187 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:34.187 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121227+0000", 2024-04-25T20:10:34.187 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2820000000000001e-06 2024-04-25T20:10:34.187 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.187 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.187 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:34.187 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123649+0000", 2024-04-25T20:10:34.188 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0024221970000000001 2024-04-25T20:10:34.188 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.188 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.188 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:34.188 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123723+0000", 2024-04-25T20:10:34.188 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.4386000000000002e-05 2024-04-25T20:10:34.188 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.188 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.188 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:34.188 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123844+0000", 2024-04-25T20:10:34.188 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000121249 2024-04-25T20:10:34.188 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.188 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.188 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:34.188 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125670+0000", 2024-04-25T20:10:34.189 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001825901 2024-04-25T20:10:34.189 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.189 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.189 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:34.189 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132043+0000", 2024-04-25T20:10:34.189 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0063724539999999996 2024-04-25T20:10:34.189 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.189 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.189 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:34.189 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132084+0000", 2024-04-25T20:10:34.189 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0963000000000001e-05 2024-04-25T20:10:34.189 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.189 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.189 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:34.189 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132114+0000", 2024-04-25T20:10:34.190 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9927e-05 2024-04-25T20:10:34.190 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.190 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:34.190 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.190 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.190 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.190 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:136 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:34.190 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:10:34.190 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 76.011630681, 2024-04-25T20:10:34.190 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010952513000000001, 2024-04-25T20:10:34.190 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:34.190 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:34.190 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:34.190 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:34.190 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:34.191 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 136 2024-04-25T20:10:34.191 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.191 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:34.191 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.191 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:34.191 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:10:34.191 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.191 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.191 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.191 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:34.191 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:10:34.191 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.191 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.191 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.191 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:34.192 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121230+0000", 2024-04-25T20:10:34.192 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:34.192 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.192 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.192 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:34.192 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121236+0000", 2024-04-25T20:10:34.192 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.7320000000000003e-06 2024-04-25T20:10:34.192 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.192 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.192 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:34.192 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121242+0000", 2024-04-25T20:10:34.192 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.7810000000000002e-06 2024-04-25T20:10:34.192 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.192 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.192 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:34.193 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121246+0000", 2024-04-25T20:10:34.193 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.639e-06 2024-04-25T20:10:34.193 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.193 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.193 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:34.193 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124150+0000", 2024-04-25T20:10:34.193 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.002903816 2024-04-25T20:10:34.193 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.193 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.193 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:34.193 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124195+0000", 2024-04-25T20:10:34.193 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.5042e-05 2024-04-25T20:10:34.193 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.193 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.193 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:34.193 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124266+0000", 2024-04-25T20:10:34.193 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0234999999999994e-05 2024-04-25T20:10:34.194 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.194 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.194 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:34.194 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125711+0000", 2024-04-25T20:10:34.194 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0014458800000000001 2024-04-25T20:10:34.194 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.194 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.194 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:34.194 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132120+0000", 2024-04-25T20:10:34.194 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0064084399999999996 2024-04-25T20:10:34.194 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.194 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.194 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:34.194 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132155+0000", 2024-04-25T20:10:34.194 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4938000000000003e-05 2024-04-25T20:10:34.195 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.195 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.195 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:34.195 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132183+0000", 2024-04-25T20:10:34.195 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8360999999999998e-05 2024-04-25T20:10:34.195 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.195 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:34.195 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.195 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.195 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.195 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:34.195 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:34.195 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 76.011580949000006, 2024-04-25T20:10:34.195 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.01097684, 2024-04-25T20:10:34.195 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:34.196 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:34.196 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:34.196 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:34.196 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:34.196 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 137 2024-04-25T20:10:34.196 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.196 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:34.196 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.196 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:34.196 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:34.196 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.196 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.196 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.196 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:34.196 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:34.196 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.197 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.197 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.197 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:34.197 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:34.197 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:34.197 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.197 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.197 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:34.197 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121328+0000", 2024-04-25T20:10:34.197 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.7908999999999998e-05 2024-04-25T20:10:34.197 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.197 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.197 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:34.197 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121337+0000", 2024-04-25T20:10:34.198 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.9369999999999993e-06 2024-04-25T20:10:34.198 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.198 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.198 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:34.198 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121343+0000", 2024-04-25T20:10:34.198 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.8089999999999998e-06 2024-04-25T20:10:34.198 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.198 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.198 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:34.198 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124656+0000", 2024-04-25T20:10:34.198 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0033133839999999999 2024-04-25T20:10:34.198 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.198 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.198 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:34.199 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124730+0000", 2024-04-25T20:10:34.199 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3565000000000005e-05 2024-04-25T20:10:34.199 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.199 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.199 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:34.199 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124846+0000", 2024-04-25T20:10:34.199 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000116416 2024-04-25T20:10:34.199 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.199 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.199 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:34.199 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126756+0000", 2024-04-25T20:10:34.199 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001909991 2024-04-25T20:10:34.199 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.199 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.200 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:34.200 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132190+0000", 2024-04-25T20:10:34.200 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0054343880000000001 2024-04-25T20:10:34.200 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.200 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.200 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:34.200 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132227+0000", 2024-04-25T20:10:34.200 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6875000000000002e-05 2024-04-25T20:10:34.200 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.200 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.200 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:34.200 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132257+0000", 2024-04-25T20:10:34.200 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.0046e-05 2024-04-25T20:10:34.200 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.200 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:34.201 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.201 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.201 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.201 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:34.201 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:34.201 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 76.011510990999994, 2024-04-25T20:10:34.201 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010978653999999999, 2024-04-25T20:10:34.201 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:34.201 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:34.201 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:34.201 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:34.201 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:34.201 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 138 2024-04-25T20:10:34.201 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.202 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:34.202 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.202 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:34.202 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:34.202 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.202 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.202 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.202 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:34.202 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:34.202 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.202 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.202 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.202 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:34.202 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:34.203 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:34.203 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.203 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.203 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:34.203 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121355+0000", 2024-04-25T20:10:34.203 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.9869999999999999e-06 2024-04-25T20:10:34.203 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.203 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.203 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:34.203 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121361+0000", 2024-04-25T20:10:34.203 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.4620000000000001e-06 2024-04-25T20:10:34.203 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.203 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.203 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:34.203 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121373+0000", 2024-04-25T20:10:34.204 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.2035000000000001e-05 2024-04-25T20:10:34.204 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.204 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.204 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:34.204 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125267+0000", 2024-04-25T20:10:34.204 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0038939230000000001 2024-04-25T20:10:34.204 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.204 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.204 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:34.204 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125309+0000", 2024-04-25T20:10:34.204 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1690999999999998e-05 2024-04-25T20:10:34.204 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.204 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.205 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:34.205 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125385+0000", 2024-04-25T20:10:34.205 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.6162999999999996e-05 2024-04-25T20:10:34.205 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.205 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.205 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:34.205 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126798+0000", 2024-04-25T20:10:34.205 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0014124680000000001 2024-04-25T20:10:34.205 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.205 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.205 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:34.205 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132264+0000", 2024-04-25T20:10:34.205 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0054661689999999999 2024-04-25T20:10:34.205 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.205 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.206 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:34.206 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132299+0000", 2024-04-25T20:10:34.206 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5272999999999999e-05 2024-04-25T20:10:34.206 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.206 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.206 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:34.206 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132329+0000", 2024-04-25T20:10:34.206 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.0080999999999999e-05 2024-04-25T20:10:34.206 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.206 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:34.206 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.206 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.206 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.206 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:34.207 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:10:34.207 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 76.011480415999998, 2024-04-25T20:10:34.207 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011015853000000001, 2024-04-25T20:10:34.207 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:34.207 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:34.207 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:34.207 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:34.207 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:34.207 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 139 2024-04-25T20:10:34.207 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.207 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:34.207 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.207 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:34.207 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:10:34.207 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.208 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.208 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.208 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:34.208 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:10:34.208 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.208 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.208 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.208 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:34.208 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121380+0000", 2024-04-25T20:10:34.208 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:34.208 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.208 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.208 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:34.208 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121385+0000", 2024-04-25T20:10:34.209 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.092e-06 2024-04-25T20:10:34.209 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.209 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.209 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:34.209 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121390+0000", 2024-04-25T20:10:34.209 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.5990000000000001e-06 2024-04-25T20:10:34.209 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.209 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.209 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:34.209 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121397+0000", 2024-04-25T20:10:34.209 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.6279999999999999e-06 2024-04-25T20:10:34.209 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.209 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.210 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:34.210 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125737+0000", 2024-04-25T20:10:34.210 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0043406010000000004 2024-04-25T20:10:34.210 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.210 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.210 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:34.210 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125781+0000", 2024-04-25T20:10:34.210 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.3958999999999997e-05 2024-04-25T20:10:34.210 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.210 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.210 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:34.210 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125849+0000", 2024-04-25T20:10:34.210 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8071999999999995e-05 2024-04-25T20:10:34.210 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.211 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.211 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:34.211 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.130908+0000", 2024-04-25T20:10:34.211 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0050586509999999999 2024-04-25T20:10:34.211 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.211 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.211 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:34.211 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132334+0000", 2024-04-25T20:10:34.211 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00142621 2024-04-25T20:10:34.211 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.211 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.211 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:34.211 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132369+0000", 2024-04-25T20:10:34.211 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4452000000000002e-05 2024-04-25T20:10:34.211 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.211 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.212 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:34.212 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132397+0000", 2024-04-25T20:10:34.212 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8065e-05 2024-04-25T20:10:34.212 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.212 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:34.212 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.212 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.212 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.212 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:34.212 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:34.212 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 76.011459372000004, 2024-04-25T20:10:34.212 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011069443, 2024-04-25T20:10:34.212 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:34.212 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:34.212 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:34.212 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:34.212 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:34.213 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 140 2024-04-25T20:10:34.213 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.213 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:34.213 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.213 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:34.213 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:34.213 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.213 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.213 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.213 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:34.213 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:34.213 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.213 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.213 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.213 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:34.213 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:34.214 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:34.214 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.214 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.214 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:34.214 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121456+0000", 2024-04-25T20:10:34.214 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.4469000000000003e-05 2024-04-25T20:10:34.214 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.214 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.214 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:34.214 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121462+0000", 2024-04-25T20:10:34.214 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.1890000000000004e-06 2024-04-25T20:10:34.214 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.214 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.214 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:34.214 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121466+0000", 2024-04-25T20:10:34.214 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3909999999999998e-06 2024-04-25T20:10:34.215 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.215 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.215 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:34.215 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126071+0000", 2024-04-25T20:10:34.215 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0046048979999999996 2024-04-25T20:10:34.215 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.215 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.215 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:34.215 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126141+0000", 2024-04-25T20:10:34.215 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0621e-05 2024-04-25T20:10:34.215 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.215 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.215 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:34.215 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126260+0000", 2024-04-25T20:10:34.215 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000118621 2024-04-25T20:10:34.216 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.216 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.216 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:34.216 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.131270+0000", 2024-04-25T20:10:34.216 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0050096949999999998 2024-04-25T20:10:34.216 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.216 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.216 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:34.216 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132403+0000", 2024-04-25T20:10:34.216 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011332779999999999 2024-04-25T20:10:34.216 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.216 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.216 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:34.216 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132443+0000", 2024-04-25T20:10:34.216 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0175000000000001e-05 2024-04-25T20:10:34.216 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.216 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.217 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:34.217 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132471+0000", 2024-04-25T20:10:34.217 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8399e-05 2024-04-25T20:10:34.217 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.217 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:34.217 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.217 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.217 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.217 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:34.217 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:10:34.217 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 76.011390586999994, 2024-04-25T20:10:34.217 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011071002, 2024-04-25T20:10:34.217 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:34.217 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:34.217 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:34.217 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:34.217 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:34.218 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 141 2024-04-25T20:10:34.218 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.218 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:34.218 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.218 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:34.218 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:10:34.218 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.218 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.218 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.218 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:34.218 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:10:34.218 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.218 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.218 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.218 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:34.218 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121470+0000", 2024-04-25T20:10:34.219 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:34.219 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.219 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.219 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:34.219 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121473+0000", 2024-04-25T20:10:34.219 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.01e-06 2024-04-25T20:10:34.219 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.219 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.219 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:34.219 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121478+0000", 2024-04-25T20:10:34.219 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.515e-06 2024-04-25T20:10:34.219 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.219 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.219 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:34.219 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121480+0000", 2024-04-25T20:10:34.219 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5749999999999999e-06 2024-04-25T20:10:34.220 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.220 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.220 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:34.220 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126813+0000", 2024-04-25T20:10:34.220 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0053320399999999997 2024-04-25T20:10:34.220 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.220 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.220 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:34.220 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126854+0000", 2024-04-25T20:10:34.220 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1582999999999998e-05 2024-04-25T20:10:34.220 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.220 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.220 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:34.220 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126923+0000", 2024-04-25T20:10:34.221 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8544e-05 2024-04-25T20:10:34.221 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.221 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.221 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:34.221 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.131573+0000", 2024-04-25T20:10:34.221 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0046504659999999998 2024-04-25T20:10:34.221 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.221 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.221 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:34.221 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132478+0000", 2024-04-25T20:10:34.221 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00090442400000000003 2024-04-25T20:10:34.221 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.221 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.221 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:34.221 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132513+0000", 2024-04-25T20:10:34.222 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5204999999999999e-05 2024-04-25T20:10:34.222 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.222 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.222 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:34.222 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132542+0000", 2024-04-25T20:10:34.222 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.904e-05 2024-04-25T20:10:34.222 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.222 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:34.222 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.222 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.222 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.222 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:156 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:34.222 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:34.222 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 75.513566291999993, 2024-04-25T20:10:34.222 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090072280000000008, 2024-04-25T20:10:34.222 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:34.223 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:34.223 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:34.223 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:34.223 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:34.223 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 156 2024-04-25T20:10:34.223 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.223 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:34.223 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.223 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:34.223 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:34.223 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.223 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.223 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.224 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:34.224 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:34.224 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.224 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.224 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.224 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:34.224 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:34.224 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:34.224 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.224 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.224 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:34.224 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619297+0000", 2024-04-25T20:10:34.224 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2979999999999999e-06 2024-04-25T20:10:34.224 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.224 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.224 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:34.225 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619300+0000", 2024-04-25T20:10:34.225 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.371e-06 2024-04-25T20:10:34.225 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.225 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.225 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:34.225 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619304+0000", 2024-04-25T20:10:34.225 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.304e-06 2024-04-25T20:10:34.225 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.225 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.225 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:34.225 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.622910+0000", 2024-04-25T20:10:34.225 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0036065049999999999 2024-04-25T20:10:34.225 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.225 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.225 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:34.225 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.622990+0000", 2024-04-25T20:10:34.226 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.9722999999999995e-05 2024-04-25T20:10:34.226 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.226 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.226 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:34.226 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623128+0000", 2024-04-25T20:10:34.226 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00013818399999999999 2024-04-25T20:10:34.226 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.226 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.226 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:34.226 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625155+0000", 2024-04-25T20:10:34.226 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0020267979999999998 2024-04-25T20:10:34.226 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.226 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.226 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:34.227 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628225+0000", 2024-04-25T20:10:34.227 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0030705680000000001 2024-04-25T20:10:34.227 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.227 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.227 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:34.227 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628268+0000", 2024-04-25T20:10:34.227 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.2404000000000003e-05 2024-04-25T20:10:34.227 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.227 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.227 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:34.227 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628302+0000", 2024-04-25T20:10:34.227 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4439e-05 2024-04-25T20:10:34.227 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.227 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:34.227 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.227 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.228 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.228 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:157 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:34.228 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:34.228 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 75.513554193000004, 2024-04-25T20:10:34.228 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090534889999999996, 2024-04-25T20:10:34.228 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:34.228 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:34.228 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:34.228 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:34.228 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:34.228 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 157 2024-04-25T20:10:34.228 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.228 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:34.228 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.228 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:34.228 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:34.229 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.229 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.229 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.229 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:34.229 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:34.229 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.229 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.229 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.229 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:34.229 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:34.229 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:34.229 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.229 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.229 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:34.229 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619309+0000", 2024-04-25T20:10:34.229 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.841e-06 2024-04-25T20:10:34.230 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.230 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.230 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:34.230 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619312+0000", 2024-04-25T20:10:34.230 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.084e-06 2024-04-25T20:10:34.230 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.230 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.230 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:34.230 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619314+0000", 2024-04-25T20:10:34.230 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.3860000000000001e-06 2024-04-25T20:10:34.230 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.230 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.230 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:34.230 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623718+0000", 2024-04-25T20:10:34.231 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0044038050000000002 2024-04-25T20:10:34.231 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.231 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.231 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:34.231 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623791+0000", 2024-04-25T20:10:34.231 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3181000000000003e-05 2024-04-25T20:10:34.231 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.231 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.231 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:34.231 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623922+0000", 2024-04-25T20:10:34.231 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00013117000000000001 2024-04-25T20:10:34.231 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.231 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.231 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:34.231 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625187+0000", 2024-04-25T20:10:34.231 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001264961 2024-04-25T20:10:34.231 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.232 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.232 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:34.232 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628308+0000", 2024-04-25T20:10:34.232 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.003120739 2024-04-25T20:10:34.232 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.232 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.232 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:34.232 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628331+0000", 2024-04-25T20:10:34.232 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2802999999999998e-05 2024-04-25T20:10:34.232 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.232 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.232 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:34.232 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628361+0000", 2024-04-25T20:10:34.232 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9760999999999998e-05 2024-04-25T20:10:34.232 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.232 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:34.233 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.233 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.233 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.233 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:158 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:34.233 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:10:34.233 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 75.513543745999996, 2024-04-25T20:10:34.233 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0091076409999999997, 2024-04-25T20:10:34.233 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:34.233 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:34.233 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:34.233 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:34.233 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:34.233 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 158 2024-04-25T20:10:34.233 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.233 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:34.233 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.233 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:34.233 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:10:34.234 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.234 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.234 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.234 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:34.234 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:10:34.234 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.234 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.234 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.234 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:34.234 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619317+0000", 2024-04-25T20:10:34.234 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967296 2024-04-25T20:10:34.234 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.234 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.234 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:34.234 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619364+0000", 2024-04-25T20:10:34.234 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.6572000000000001e-05 2024-04-25T20:10:34.235 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.235 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.235 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:34.235 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619371+0000", 2024-04-25T20:10:34.235 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3860000000000001e-06 2024-04-25T20:10:34.235 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.235 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.235 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:34.235 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619382+0000", 2024-04-25T20:10:34.235 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0173999999999999e-05 2024-04-25T20:10:34.235 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.235 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.235 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:34.235 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624216+0000", 2024-04-25T20:10:34.236 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048340350000000004 2024-04-25T20:10:34.236 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.236 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.236 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:34.236 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624260+0000", 2024-04-25T20:10:34.236 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4558999999999998e-05 2024-04-25T20:10:34.236 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.236 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.236 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:34.236 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624333+0000", 2024-04-25T20:10:34.236 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2744999999999996e-05 2024-04-25T20:10:34.236 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.236 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.236 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:34.236 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625465+0000", 2024-04-25T20:10:34.237 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001132454 2024-04-25T20:10:34.237 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.237 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.237 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:34.237 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628365+0000", 2024-04-25T20:10:34.237 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0028994950000000002 2024-04-25T20:10:34.237 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.237 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.237 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:34.237 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628386+0000", 2024-04-25T20:10:34.237 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.1367000000000001e-05 2024-04-25T20:10:34.237 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.237 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.237 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:34.237 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628425+0000", 2024-04-25T20:10:34.237 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.9087e-05 2024-04-25T20:10:34.238 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.238 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:34.238 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.238 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.238 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.238 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:159 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:34.238 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:34.238 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 75.513475092999997, 2024-04-25T20:10:34.238 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090948999999999995, 2024-04-25T20:10:34.238 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:34.238 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:34.238 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:34.238 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:34.238 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:34.239 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 159 2024-04-25T20:10:34.239 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.239 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:34.239 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.239 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:34.239 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:34.239 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.239 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.239 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.239 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:34.239 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:34.239 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:34.239 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.239 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.239 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:34.240 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:34.240 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:34.240 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.240 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.240 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:34.240 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619389+0000", 2024-04-25T20:10:34.240 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.734e-06 2024-04-25T20:10:34.240 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.240 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.240 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:34.240 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619392+0000", 2024-04-25T20:10:34.240 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6219999999999998e-06 2024-04-25T20:10:34.240 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.240 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.240 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:34.241 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619403+0000", 2024-04-25T20:10:34.241 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0987e-05 2024-04-25T20:10:34.241 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.241 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.241 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:34.241 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624667+0000", 2024-04-25T20:10:34.241 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0052633970000000004 2024-04-25T20:10:34.241 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.241 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.241 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:34.241 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624741+0000", 2024-04-25T20:10:34.241 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.4583000000000001e-05 2024-04-25T20:10:34.241 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.241 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.241 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:34.241 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624857+0000", 2024-04-25T20:10:34.242 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000116118 2024-04-25T20:10:34.242 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.242 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.242 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:34.242 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.626013+0000", 2024-04-25T20:10:34.242 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001155473 2024-04-25T20:10:34.242 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.242 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.242 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:34.242 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628432+0000", 2024-04-25T20:10:34.242 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0024189379999999998 2024-04-25T20:10:34.242 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.242 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.242 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:34.242 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628454+0000", 2024-04-25T20:10:34.242 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2059999999999999e-05 2024-04-25T20:10:34.243 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:34.243 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:34.243 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:34.243 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628481+0000", 2024-04-25T20:10:34.243 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.7300999999999998e-05 2024-04-25T20:10:34.243 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.243 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:34.243 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.243 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:34.243 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:34.243 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:34.244 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:34.244+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:34.245 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T20:10:34.345 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:34.345+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:34.352 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:34.352 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:34.352 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:34.352 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:34.362 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T20:10:34.446 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:34.446+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:34.522 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:34.522 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:34.522 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:34.522 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:34.523 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:34.532 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T20:10:34.546 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:34.546+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:34.639 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:34.639 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:34.639 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:34.639 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:10:34.639 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:34.648 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:34.647+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:34.649 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T20:10:34.748 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:34.748+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:34.755 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:34.755 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:34.755 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:34.755 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:34.765 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T20:10:34.849 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:34.849+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:34.926 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:34.926 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:34.926 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:34.926 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:34.926 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:34.938 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T20:10:34.979 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:34.979+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:35.051 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:35.050+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:35.088 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:35.089 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:35.089 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:35.089 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-04-25T20:10:35.089 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.089 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:30 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'16, mlcod=16'14)", 2024-04-25T20:10:35.089 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:10:35.089 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 77.920455833999995, 2024-04-25T20:10:35.089 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0054087670000000001, 2024-04-25T20:10:35.089 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:35.089 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:35.089 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:35.089 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.089 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:35.089 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:10:35.089 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.090 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.090 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.090 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:35.090 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:10:35.090 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.090 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.090 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.090 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:35.090 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167142+0000", 2024-04-25T20:10:35.090 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999976 2024-04-25T20:10:35.090 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.090 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.090 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:35.090 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167162+0000", 2024-04-25T20:10:35.090 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0109e-05 2024-04-25T20:10:35.091 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.091 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.091 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:35.091 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167188+0000", 2024-04-25T20:10:35.091 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6146e-05 2024-04-25T20:10:35.091 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.091 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.091 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:35.091 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167199+0000", 2024-04-25T20:10:35.091 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0767e-05 2024-04-25T20:10:35.091 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.091 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.091 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:35.091 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167256+0000", 2024-04-25T20:10:35.091 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7392000000000002e-05 2024-04-25T20:10:35.091 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.091 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.092 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:35.092 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167283+0000", 2024-04-25T20:10:35.092 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6577e-05 2024-04-25T20:10:35.092 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.092 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.092 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:35.092 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.172510+0000", 2024-04-25T20:10:35.092 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0052276759999999997 2024-04-25T20:10:35.092 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.092 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.092 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:35.092 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.172553+0000", 2024-04-25T20:10:35.092 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2444999999999998e-05 2024-04-25T20:10:35.093 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.093 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:35.093 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.093 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.093 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.093 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:31 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-04-25T20:10:35.093 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:35.093 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 77.919839756000002, 2024-04-25T20:10:35.093 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0055055479999999999, 2024-04-25T20:10:35.093 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:35.093 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:35.093 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:35.093 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.093 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:35.093 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:35.094 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.094 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.094 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.094 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:35.094 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:35.094 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.094 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.094 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.094 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:35.094 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:35.094 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:35.094 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.094 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.095 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:35.095 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167766+0000", 2024-04-25T20:10:35.095 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.122e-06 2024-04-25T20:10:35.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.095 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.095 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:35.095 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167776+0000", 2024-04-25T20:10:35.095 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0515e-05 2024-04-25T20:10:35.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.095 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.095 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:35.095 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167782+0000", 2024-04-25T20:10:35.095 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1149999999999999e-06 2024-04-25T20:10:35.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.095 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.096 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:35.096 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167837+0000", 2024-04-25T20:10:35.096 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4763999999999999e-05 2024-04-25T20:10:35.096 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.096 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.096 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:35.096 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167855+0000", 2024-04-25T20:10:35.096 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7589e-05 2024-04-25T20:10:35.096 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.096 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.096 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:35.096 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173221+0000", 2024-04-25T20:10:35.096 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.005365903 2024-04-25T20:10:35.096 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.096 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.096 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:35.097 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173266+0000", 2024-04-25T20:10:35.097 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5059999999999999e-05 2024-04-25T20:10:35.097 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.097 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:35.097 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.097 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.097 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:32 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-04-25T20:10:35.097 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:35.097 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 77.919464571000006, 2024-04-25T20:10:35.097 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0051558239999999998, 2024-04-25T20:10:35.098 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:35.098 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:35.098 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:35.098 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.098 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:35.098 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:35.098 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.098 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.098 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.098 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:35.098 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:35.098 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.099 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.099 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.099 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:35.099 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:35.099 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:35.099 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.099 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.099 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:35.099 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168173+0000", 2024-04-25T20:10:35.099 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7663999999999997e-05 2024-04-25T20:10:35.099 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.099 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.099 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:35.099 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168182+0000", 2024-04-25T20:10:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0650000000000005e-06 2024-04-25T20:10:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168188+0000", 2024-04-25T20:10:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0589999999999999e-06 2024-04-25T20:10:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168239+0000", 2024-04-25T20:10:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1467e-05 2024-04-25T20:10:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:35.101 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168260+0000", 2024-04-25T20:10:35.101 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0531e-05 2024-04-25T20:10:35.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.101 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.101 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:35.101 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173268+0000", 2024-04-25T20:10:35.101 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0050087980000000001 2024-04-25T20:10:35.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.101 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.101 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:35.101 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173291+0000", 2024-04-25T20:10:35.101 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.268e-05 2024-04-25T20:10:35.101 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.101 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:35.102 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.102 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.102 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:33 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'19, mlcod=16'14)", 2024-04-25T20:10:35.102 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:10:35.102 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 77.918954151999998, 2024-04-25T20:10:35.102 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046697049999999997, 2024-04-25T20:10:35.102 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:35.102 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:35.102 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:35.102 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.102 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:35.102 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:10:35.102 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.103 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.103 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.103 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:35.103 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:10:35.103 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.103 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.103 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.103 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:35.103 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168645+0000", 2024-04-25T20:10:35.103 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:35.103 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.103 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.103 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:35.104 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168651+0000", 2024-04-25T20:10:35.104 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1079999999999998e-06 2024-04-25T20:10:35.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.104 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.104 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:35.104 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168660+0000", 2024-04-25T20:10:35.104 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0010000000000007e-06 2024-04-25T20:10:35.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.104 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.104 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:35.104 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168666+0000", 2024-04-25T20:10:35.104 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7780000000000004e-06 2024-04-25T20:10:35.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.104 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168701+0000", 2024-04-25T20:10:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4490999999999999e-05 2024-04-25T20:10:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168729+0000", 2024-04-25T20:10:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8476999999999999e-05 2024-04-25T20:10:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173294+0000", 2024-04-25T20:10:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0045650730000000002 2024-04-25T20:10:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.106 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:35.106 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173315+0000", 2024-04-25T20:10:35.106 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1245000000000001e-05 2024-04-25T20:10:35.106 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.106 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:35.106 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.106 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.106 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:34 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'20, mlcod=16'14)", 2024-04-25T20:10:35.106 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:10:35.106 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 77.918455222999995, 2024-04-25T20:10:35.106 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041943309999999999, 2024-04-25T20:10:35.106 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:35.106 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:10:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:10:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169144+0000", 2024-04-25T20:10:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.108 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.108 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:35.108 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169150+0000", 2024-04-25T20:10:35.108 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9150000000000001e-06 2024-04-25T20:10:35.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.108 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.108 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:35.108 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169159+0000", 2024-04-25T20:10:35.108 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8899999999999996e-06 2024-04-25T20:10:35.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.108 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.108 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:35.108 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169169+0000", 2024-04-25T20:10:35.108 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0441e-05 2024-04-25T20:10:35.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.108 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.108 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:35.109 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169206+0000", 2024-04-25T20:10:35.109 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6816000000000001e-05 2024-04-25T20:10:35.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.109 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.109 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:35.109 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169234+0000", 2024-04-25T20:10:35.109 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8008999999999999e-05 2024-04-25T20:10:35.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.109 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.109 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:35.109 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173317+0000", 2024-04-25T20:10:35.109 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0040830160000000001 2024-04-25T20:10:35.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.109 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.109 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:35.110 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173339+0000", 2024-04-25T20:10:35.110 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1897000000000001e-05 2024-04-25T20:10:35.110 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.110 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:35.110 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.110 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.110 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.110 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:35 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'14)", 2024-04-25T20:10:35.110 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:10:35.110 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 77.917962904000007, 2024-04-25T20:10:35.110 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037236529999999999, 2024-04-25T20:10:35.110 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:35.110 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:35.110 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:35.110 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.110 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:35.111 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:10:35.111 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.111 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.111 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:35.111 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:10:35.111 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.111 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.111 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:35.111 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169636+0000", 2024-04-25T20:10:35.111 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:35.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.111 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.111 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:35.111 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169678+0000", 2024-04-25T20:10:35.111 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1752000000000003e-05 2024-04-25T20:10:35.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.112 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.112 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:35.112 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169688+0000", 2024-04-25T20:10:35.112 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0098000000000001e-05 2024-04-25T20:10:35.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.112 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.112 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:35.112 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169694+0000", 2024-04-25T20:10:35.112 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2349999999999996e-06 2024-04-25T20:10:35.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.112 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.112 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:35.112 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169733+0000", 2024-04-25T20:10:35.112 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8481e-05 2024-04-25T20:10:35.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.112 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169750+0000", 2024-04-25T20:10:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7509000000000001e-05 2024-04-25T20:10:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173340+0000", 2024-04-25T20:10:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0035898449999999999 2024-04-25T20:10:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173361+0000", 2024-04-25T20:10:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.039e-05 2024-04-25T20:10:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.114 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.114 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:36 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'22, mlcod=16'14)", 2024-04-25T20:10:35.114 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:35.114 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 77.917462843999999, 2024-04-25T20:10:35.114 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032472120000000002, 2024-04-25T20:10:35.114 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:35.114 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:35.114 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:35.114 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.114 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:35.114 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:35.114 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.114 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170152+0000", 2024-04-25T20:10:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5798e-05 2024-04-25T20:10:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170161+0000", 2024-04-25T20:10:35.116 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.064e-06 2024-04-25T20:10:35.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.116 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.116 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:35.116 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170167+0000", 2024-04-25T20:10:35.116 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8390000000000003e-06 2024-04-25T20:10:35.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.116 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.116 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:35.116 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170202+0000", 2024-04-25T20:10:35.116 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.5120999999999998e-05 2024-04-25T20:10:35.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.116 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.116 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:35.116 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170219+0000", 2024-04-25T20:10:35.116 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6929999999999999e-05 2024-04-25T20:10:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173362+0000", 2024-04-25T20:10:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0031427619999999999 2024-04-25T20:10:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173384+0000", 2024-04-25T20:10:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2237000000000001e-05 2024-04-25T20:10:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:40 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'26, mlcod=16'24)", 2024-04-25T20:10:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:10:35.118 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 77.910424931999998, 2024-04-25T20:10:35.118 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028727330000000001, 2024-04-25T20:10:35.118 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:35.118 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:35.118 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:35.118 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.118 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:35.118 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:10:35.118 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.118 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.118 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:35.118 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:10:35.118 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.118 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177173+0000", 2024-04-25T20:10:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999976 2024-04-25T20:10:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177186+0000", 2024-04-25T20:10:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3533000000000001e-05 2024-04-25T20:10:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177209+0000", 2024-04-25T20:10:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3280000000000001e-05 2024-04-25T20:10:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177220+0000", 2024-04-25T20:10:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0465000000000001e-05 2024-04-25T20:10:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177264+0000", 2024-04-25T20:10:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4413999999999999e-05 2024-04-25T20:10:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177289+0000", 2024-04-25T20:10:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4783999999999999e-05 2024-04-25T20:10:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.179990+0000", 2024-04-25T20:10:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027012939999999999 2024-04-25T20:10:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.121 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.121 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:35.121 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.180048+0000", 2024-04-25T20:10:35.121 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7299000000000001e-05 2024-04-25T20:10:35.121 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.121 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:35.121 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.121 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.122 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:67 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'41, mlcod=16'39)", 2024-04-25T20:10:35.122 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:10:35.122 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 77.875663439999997, 2024-04-25T20:10:35.122 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064192549999999996, 2024-04-25T20:10:35.122 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:35.122 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:35.123 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:35.123 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.123 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:35.123 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:10:35.123 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.123 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.123 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:35.123 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:10:35.123 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.123 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.123 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:35.123 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211935+0000", 2024-04-25T20:10:35.123 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999981 2024-04-25T20:10:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211948+0000", 2024-04-25T20:10:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3531e-05 2024-04-25T20:10:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211967+0000", 2024-04-25T20:10:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8326999999999999e-05 2024-04-25T20:10:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211975+0000", 2024-04-25T20:10:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8589999999999994e-06 2024-04-25T20:10:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212020+0000", 2024-04-25T20:10:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4626999999999998e-05 2024-04-25T20:10:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212045+0000", 2024-04-25T20:10:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4508999999999999e-05 2024-04-25T20:10:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218316+0000", 2024-04-25T20:10:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0062711640000000001 2024-04-25T20:10:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.126 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:35.126 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218356+0000", 2024-04-25T20:10:35.126 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9993000000000001e-05 2024-04-25T20:10:35.126 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.126 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:35.126 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.126 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.126 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:68 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-04-25T20:10:35.126 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:35.126 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 77.875214425999999, 2024-04-25T20:10:35.126 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011231365, 2024-04-25T20:10:35.126 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:35.126 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:35.126 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.128 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.128 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:35.128 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212392+0000", 2024-04-25T20:10:35.128 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.79e-06 2024-04-25T20:10:35.128 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.128 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.128 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:35.128 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212406+0000", 2024-04-25T20:10:35.128 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3577000000000001e-05 2024-04-25T20:10:35.128 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.128 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.128 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:35.128 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212414+0000", 2024-04-25T20:10:35.128 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6510000000000001e-06 2024-04-25T20:10:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217114+0000", 2024-04-25T20:10:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047005379999999998 2024-04-25T20:10:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217141+0000", 2024-04-25T20:10:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6984000000000001e-05 2024-04-25T20:10:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223563+0000", 2024-04-25T20:10:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064216999999999998 2024-04-25T20:10:35.130 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.130 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.130 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:35.131 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223617+0000", 2024-04-25T20:10:35.131 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4011000000000002e-05 2024-04-25T20:10:35.131 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.131 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:35.131 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.132 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.132 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.132 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:69 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-04-25T20:10:35.132 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:10:35.132 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 77.874682235999998, 2024-04-25T20:10:35.132 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.010730094000000001, 2024-04-25T20:10:35.132 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:35.132 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:35.132 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:35.133 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.133 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:35.133 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:10:35.133 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.133 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.133 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:35.133 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:10:35.133 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.133 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.134 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:35.134 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212917+0000", 2024-04-25T20:10:35.134 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:35.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.134 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.134 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:35.134 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212957+0000", 2024-04-25T20:10:35.134 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0692000000000003e-05 2024-04-25T20:10:35.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.134 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.134 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212973+0000", 2024-04-25T20:10:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.575e-05 2024-04-25T20:10:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212980+0000", 2024-04-25T20:10:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1679999999999998e-06 2024-04-25T20:10:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217449+0000", 2024-04-25T20:10:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.004468367 2024-04-25T20:10:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217470+0000", 2024-04-25T20:10:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0749000000000001e-05 2024-04-25T20:10:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223621+0000", 2024-04-25T20:10:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0061512880000000004 2024-04-25T20:10:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223648+0000", 2024-04-25T20:10:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6961000000000002e-05 2024-04-25T20:10:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:70 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'44, mlcod=16'39)", 2024-04-25T20:10:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 77.873923508000004, 2024-04-25T20:10:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.013553358, 2024-04-25T20:10:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:35.138 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:35.138 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.138 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.138 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:35.138 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:35.138 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:35.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.138 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.138 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:35.138 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213682+0000", 2024-04-25T20:10:35.138 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3439999999999997e-06 2024-04-25T20:10:35.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.139 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.139 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:35.139 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213693+0000", 2024-04-25T20:10:35.139 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1049999999999999e-05 2024-04-25T20:10:35.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.139 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.139 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:35.139 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213699+0000", 2024-04-25T20:10:35.139 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9399999999999999e-06 2024-04-25T20:10:35.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.139 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.139 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:35.139 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218086+0000", 2024-04-25T20:10:35.139 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0043871240000000001 2024-04-25T20:10:35.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.140 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.140 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:35.140 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218106+0000", 2024-04-25T20:10:35.140 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.997e-05 2024-04-25T20:10:35.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.140 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.140 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:35.140 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.227186+0000", 2024-04-25T20:10:35.140 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.009079574 2024-04-25T20:10:35.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.140 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.140 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:35.140 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.227230+0000", 2024-04-25T20:10:35.141 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4147999999999998e-05 2024-04-25T20:10:35.141 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.141 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:35.141 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.141 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.141 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:106 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-04-25T20:10:35.141 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:10:35.141 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 77.502784407999997, 2024-04-25T20:10:35.141 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002936756, 2024-04-25T20:10:35.142 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:35.142 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:35.142 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:35.142 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.142 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:35.142 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:10:35.142 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.142 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.142 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.142 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:35.142 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:10:35.143 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.143 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.143 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:35.143 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584815+0000", 2024-04-25T20:10:35.143 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:35.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.143 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.143 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:35.144 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584860+0000", 2024-04-25T20:10:35.144 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5161999999999999e-05 2024-04-25T20:10:35.144 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.144 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.144 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:35.144 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584874+0000", 2024-04-25T20:10:35.144 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4464000000000001e-05 2024-04-25T20:10:35.144 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.144 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.144 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:35.144 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584881+0000", 2024-04-25T20:10:35.144 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.409e-06 2024-04-25T20:10:35.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.145 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.145 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:35.145 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584915+0000", 2024-04-25T20:10:35.145 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4007000000000002e-05 2024-04-25T20:10:35.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.145 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.145 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:35.145 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584933+0000", 2024-04-25T20:10:35.145 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7737999999999999e-05 2024-04-25T20:10:35.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.145 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.145 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:35.145 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587717+0000", 2024-04-25T20:10:35.146 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002784711 2024-04-25T20:10:35.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.146 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.146 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:35.146 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587752+0000", 2024-04-25T20:10:35.146 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4915e-05 2024-04-25T20:10:35.146 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.146 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:35.146 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.146 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.146 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:107 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'69, mlcod=16'64)", 2024-04-25T20:10:35.146 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:10:35.146 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 77.502709105999998, 2024-04-25T20:10:35.147 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028906270000000002, 2024-04-25T20:10:35.147 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:35.147 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:35.147 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:35.147 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.147 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:35.147 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:10:35.147 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.147 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.147 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:35.147 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:10:35.148 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.148 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.148 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:35.148 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584890+0000", 2024-04-25T20:10:35.148 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:35.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.148 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.148 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:35.148 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584895+0000", 2024-04-25T20:10:35.149 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2069999999999997e-06 2024-04-25T20:10:35.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.149 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.149 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:35.149 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584906+0000", 2024-04-25T20:10:35.149 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.117e-05 2024-04-25T20:10:35.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.149 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.149 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:35.150 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584910+0000", 2024-04-25T20:10:35.150 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4190000000000004e-06 2024-04-25T20:10:35.150 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.150 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.150 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:35.150 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585310+0000", 2024-04-25T20:10:35.150 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00040002700000000001 2024-04-25T20:10:35.150 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.150 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.150 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:35.150 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585329+0000", 2024-04-25T20:10:35.151 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8838000000000002e-05 2024-04-25T20:10:35.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.151 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.151 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:35.151 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587754+0000", 2024-04-25T20:10:35.151 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024252689999999999 2024-04-25T20:10:35.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.151 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.151 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:35.151 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587781+0000", 2024-04-25T20:10:35.151 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7206999999999999e-05 2024-04-25T20:10:35.151 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.151 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:35.151 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.151 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.152 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:108 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'70, mlcod=16'64)", 2024-04-25T20:10:35.152 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:10:35.152 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 77.502683954999995, 2024-04-25T20:10:35.152 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028924879999999999, 2024-04-25T20:10:35.152 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:35.152 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:35.152 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:35.152 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.152 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:35.152 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:10:35.152 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.152 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.153 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.153 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:35.153 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:10:35.153 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.153 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.153 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:35.153 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:10:35.153 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:35.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.153 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.153 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:35.153 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584919+0000", 2024-04-25T20:10:35.154 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.557e-06 2024-04-25T20:10:35.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.154 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.154 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:35.154 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584925+0000", 2024-04-25T20:10:35.154 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8619999999999999e-06 2024-04-25T20:10:35.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.154 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.154 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:35.154 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584931+0000", 2024-04-25T20:10:35.154 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2539999999999998e-06 2024-04-25T20:10:35.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.154 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.154 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:35.155 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585550+0000", 2024-04-25T20:10:35.155 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00061821800000000002 2024-04-25T20:10:35.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.155 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.155 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:35.155 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585567+0000", 2024-04-25T20:10:35.155 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7499e-05 2024-04-25T20:10:35.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.155 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.155 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:35.155 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587783+0000", 2024-04-25T20:10:35.155 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0022156200000000002 2024-04-25T20:10:35.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.155 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.155 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:35.156 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587808+0000", 2024-04-25T20:10:35.156 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5729e-05 2024-04-25T20:10:35.156 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.156 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:35.156 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.156 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.156 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:112 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'74, mlcod=16'66)", 2024-04-25T20:10:35.156 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:10:35.156 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 77.500526332999996, 2024-04-25T20:10:35.156 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00318724, 2024-04-25T20:10:35.156 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:35.156 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:35.156 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:10:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:10:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587073+0000", 2024-04-25T20:10:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587136+0000", 2024-04-25T20:10:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2928999999999993e-05 2024-04-25T20:10:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587147+0000", 2024-04-25T20:10:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1182999999999999e-05 2024-04-25T20:10:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587153+0000", 2024-04-25T20:10:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9059999999999999e-06 2024-04-25T20:10:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.159 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.159 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:35.159 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587306+0000", 2024-04-25T20:10:35.159 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000152848 2024-04-25T20:10:35.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.159 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.159 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:35.159 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587330+0000", 2024-04-25T20:10:35.159 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4304e-05 2024-04-25T20:10:35.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.159 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.159 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:35.159 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.590230+0000", 2024-04-25T20:10:35.159 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029004629999999998 2024-04-25T20:10:35.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.160 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.160 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:35.160 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.590261+0000", 2024-04-25T20:10:35.160 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0323000000000001e-05 2024-04-25T20:10:35.160 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.160 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:35.160 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.160 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.160 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:118 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'80, mlcod=16'78)", 2024-04-25T20:10:35.160 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:10:35.161 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 77.495739753999999, 2024-04-25T20:10:35.161 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029618959999999999, 2024-04-25T20:10:35.161 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:35.161 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:35.161 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:35.161 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.161 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:35.161 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:10:35.161 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.161 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.161 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.161 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:35.161 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:10:35.162 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.162 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.162 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:35.162 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591859+0000", 2024-04-25T20:10:35.162 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:35.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.162 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.162 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:35.162 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591918+0000", 2024-04-25T20:10:35.162 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8842999999999999e-05 2024-04-25T20:10:35.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.162 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.163 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:35.163 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591933+0000", 2024-04-25T20:10:35.163 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4600000000000001e-05 2024-04-25T20:10:35.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.163 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.163 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:35.163 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591939+0000", 2024-04-25T20:10:35.163 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4559999999999997e-06 2024-04-25T20:10:35.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.163 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.163 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:35.163 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591990+0000", 2024-04-25T20:10:35.163 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0318999999999997e-05 2024-04-25T20:10:35.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.592008+0000", 2024-04-25T20:10:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8819000000000001e-05 2024-04-25T20:10:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.594784+0000", 2024-04-25T20:10:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027750850000000001 2024-04-25T20:10:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.594822+0000", 2024-04-25T20:10:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8522000000000001e-05 2024-04-25T20:10:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:139 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'97, mlcod=17'87)", 2024-04-25T20:10:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:10:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 76.961703149000002, 2024-04-25T20:10:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048546290000000001, 2024-04-25T20:10:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:10:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.166 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.166 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:35.166 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:10:35.166 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.166 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.166 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:35.166 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125896+0000", 2024-04-25T20:10:35.166 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:35.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.166 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.166 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:35.166 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125908+0000", 2024-04-25T20:10:35.166 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.137e-05 2024-04-25T20:10:35.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.167 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.167 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:35.167 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125915+0000", 2024-04-25T20:10:35.167 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3939999999999998e-06 2024-04-25T20:10:35.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.167 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.167 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:35.167 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125922+0000", 2024-04-25T20:10:35.167 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1930000000000004e-06 2024-04-25T20:10:35.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.167 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.167 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:35.167 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125959+0000", 2024-04-25T20:10:35.167 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6701e-05 2024-04-25T20:10:35.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.168 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.168 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:35.168 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125972+0000", 2024-04-25T20:10:35.168 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2578000000000001e-05 2024-04-25T20:10:35.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.168 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.168 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:35.168 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130719+0000", 2024-04-25T20:10:35.168 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047475800000000004 2024-04-25T20:10:35.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.168 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.168 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:35.168 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130751+0000", 2024-04-25T20:10:35.169 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2283000000000002e-05 2024-04-25T20:10:35.169 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.169 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:35.169 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.169 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.169 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:140 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'98, mlcod=17'87)", 2024-04-25T20:10:35.169 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:35.169 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 76.961277797999998, 2024-04-25T20:10:35.169 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048310990000000002, 2024-04-25T20:10:35.169 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:35.169 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:35.169 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:35.169 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.169 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:35.169 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:35.170 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.170 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.170 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.170 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:35.170 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:35.170 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.170 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.170 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.170 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:35.170 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:35.170 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:35.170 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.170 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.170 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:35.171 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126427+0000", 2024-04-25T20:10:35.171 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000105572 2024-04-25T20:10:35.171 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.171 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.171 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:35.171 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126435+0000", 2024-04-25T20:10:35.171 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1920000000000005e-06 2024-04-25T20:10:35.171 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.171 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.171 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:35.171 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126439+0000", 2024-04-25T20:10:35.171 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6950000000000002e-06 2024-04-25T20:10:35.171 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.171 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.172 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:35.172 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126471+0000", 2024-04-25T20:10:35.172 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1699000000000003e-05 2024-04-25T20:10:35.172 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.172 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.172 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:35.172 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126484+0000", 2024-04-25T20:10:35.172 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.343e-05 2024-04-25T20:10:35.172 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.172 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.172 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:35.172 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131131+0000", 2024-04-25T20:10:35.172 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046471480000000003 2024-04-25T20:10:35.172 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.173 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.173 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:35.173 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131153+0000", 2024-04-25T20:10:35.173 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1753e-05 2024-04-25T20:10:35.173 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.173 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:35.173 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.173 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.173 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.173 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:141 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'99, mlcod=17'87)", 2024-04-25T20:10:35.173 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:35.173 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 76.960622853999993, 2024-04-25T20:10:35.173 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.004496847, 2024-04-25T20:10:35.173 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:35.174 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:35.174 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:35.174 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.174 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:35.174 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:35.174 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.174 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.174 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.174 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:35.174 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:35.174 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:35.174 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.174 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.174 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:35.175 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:35.175 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:35.175 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.175 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.175 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:35.175 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126980+0000", 2024-04-25T20:10:35.175 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.619e-06 2024-04-25T20:10:35.175 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.175 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.175 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:35.175 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126987+0000", 2024-04-25T20:10:35.175 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7220000000000002e-06 2024-04-25T20:10:35.175 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.175 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.176 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:35.176 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126992+0000", 2024-04-25T20:10:35.176 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8679999999999998e-06 2024-04-25T20:10:35.176 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.176 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.176 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:35.176 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130772+0000", 2024-04-25T20:10:35.176 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037803120000000001 2024-04-25T20:10:35.176 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.176 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.176 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:35.176 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130786+0000", 2024-04-25T20:10:35.176 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3592e-05 2024-04-25T20:10:35.176 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.176 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.177 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:35.177 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131451+0000", 2024-04-25T20:10:35.177 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000665206 2024-04-25T20:10:35.177 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:35.177 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:35.177 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:35.177 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131474+0000", 2024-04-25T20:10:35.177 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2957999999999999e-05 2024-04-25T20:10:35.177 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.177 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:35.177 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.177 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:35.177 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:35.177 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:35.179 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T20:10:35.179 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:35.178+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:35.279 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:35.279+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:35.327 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:35.327 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:35.327 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:35.327 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:35.336 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T20:10:35.421 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:35.421+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:35.482 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:35.480+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:35.484 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:35.484 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:35.484 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:35.484 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:35.484 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:35.494 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T20:10:35.533 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T20:10:35.582 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:35.581+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:35.647 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:35.648 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:35.648 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:35.648 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:10:35.648 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:35.658 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T20:10:35.698 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:35.698+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:35.722 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:10:35.722 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:10:35.756 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:10:35.756 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:10:35.784 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:35.783+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:35.788 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:10:35.789 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:10:35.808 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:35.808 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:35.808 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:35.808 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:35.818 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T20:10:35.820 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:10:35.820 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:10:35.828 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-25T20:10:35.852 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:10:35.853 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:10:35.885 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:10:35.885 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:10:35.885 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:35.884+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:35.917 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:10:35.917 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:10:35.949 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:10:35.949 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:10:35.985 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:35.985 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:35.985 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:35.985+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:35.997 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T20:10:36.086 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:36.086+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:36.154 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:36.155 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:36.164 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T20:10:36.205 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:36.205+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:36.279 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:36.279 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:36.288 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T20:10:36.289 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:36.287+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:36.389 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:36.388+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:36.432 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:36.432 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:36.440 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T20:10:36.525 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:36.524+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:36.582 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:36.583 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:36.591 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T20:10:36.592 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:36.591+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:36.692 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:36.691+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:36.732 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:36.732 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:36.742 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T20:10:36.792 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:36.792+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:36.843 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:36.844 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:36.853 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T20:10:36.853 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is unset 2024-04-25T20:10:36.908 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:36.908+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:36.994 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:36.993+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:36.994 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:36.994 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:37.003 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T20:10:37.094 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:37.094+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:37.145 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:37.145 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:37.154 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T20:10:37.195 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:37.194+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:37.296 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:37.296 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:37.296 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:37.296+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:37.304 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T20:10:37.397 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:37.397+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:37.406 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:37.406 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:37.415 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T20:10:37.498 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:37.498+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:37.516 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:37.516 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:37.525 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T20:10:37.599 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:37.599+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:37.626 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:37.626 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:37.635 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T20:10:37.700 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:37.700+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:37.735 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:37.735 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:37.744 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T20:10:37.778 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7] out_osds: [3, 2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-25T20:10:37.778 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T20:10:37.778 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-04-25T20:10:37.778 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-04-25T20:10:37.801 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:37.801+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:37.861 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:37.861 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:37.872 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T20:10:37.915 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:37.915+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:37.967 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T20:10:38.003 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:38.002+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:38.030 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:38.030 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:38.040 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T20:10:38.104 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:38.103+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:38.176 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:10:38.176 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:10:38.190 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:38.191 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:38.191 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:38.191 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:10:38.191 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:38.200 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T20:10:38.205 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:38.204+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:38.208 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:10:38.209 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:10:38.241 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:10:38.241 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:10:38.274 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:10:38.275 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:10:38.306 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:38.305+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:38.307 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:10:38.307 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:10:38.308 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:10:38.308 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:38.308 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 54, 2024-04-25T20:10:38.308 INFO:teuthology.orchestra.run.smithi039.stdout: "fsid": "975ceafd-555d-4f71-81e4-1f70364da00f", 2024-04-25T20:10:38.308 INFO:teuthology.orchestra.run.smithi039.stdout: "created": "2024-04-25T20:09:03.377320+0000", 2024-04-25T20:10:38.308 INFO:teuthology.orchestra.run.smithi039.stdout: "modified": "2024-04-25T20:10:36.840457+0000", 2024-04-25T20:10:38.309 INFO:teuthology.orchestra.run.smithi039.stdout: "last_up_change": "2024-04-25T20:09:13.740341+0000", 2024-04-25T20:10:38.309 INFO:teuthology.orchestra.run.smithi039.stdout: "last_in_change": "2024-04-25T20:10:08.354345+0000", 2024-04-25T20:10:38.309 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T20:10:38.309 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_num": 5799936, 2024-04-25T20:10:38.309 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_set": [ 2024-04-25T20:10:38.309 INFO:teuthology.orchestra.run.smithi039.stdout: "pglog_hardlimit", 2024-04-25T20:10:38.309 INFO:teuthology.orchestra.run.smithi039.stdout: "purged_snapdirs", 2024-04-25T20:10:38.309 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_deletes", 2024-04-25T20:10:38.309 INFO:teuthology.orchestra.run.smithi039.stdout: "sortbitwise" 2024-04-25T20:10:38.309 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-04-25T20:10:38.309 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_version": 4, 2024-04-25T20:10:38.309 INFO:teuthology.orchestra.run.smithi039.stdout: "full_ratio": 0.94999998807907104, 2024-04-25T20:10:38.309 INFO:teuthology.orchestra.run.smithi039.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-04-25T20:10:38.309 INFO:teuthology.orchestra.run.smithi039.stdout: "nearfull_ratio": 0.85000002384185791, 2024-04-25T20:10:38.310 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_snapshot": "", 2024-04-25T20:10:38.310 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_max": 2, 2024-04-25T20:10:38.310 INFO:teuthology.orchestra.run.smithi039.stdout: "max_osd": 8, 2024-04-25T20:10:38.310 INFO:teuthology.orchestra.run.smithi039.stdout: "require_min_compat_client": "luminous", 2024-04-25T20:10:38.310 INFO:teuthology.orchestra.run.smithi039.stdout: "min_compat_client": "jewel", 2024-04-25T20:10:38.310 INFO:teuthology.orchestra.run.smithi039.stdout: "require_osd_release": "reef", 2024-04-25T20:10:38.310 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_crimson": false, 2024-04-25T20:10:38.310 INFO:teuthology.orchestra.run.smithi039.stdout: "pools": [ 2024-04-25T20:10:38.310 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.310 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 1, 2024-04-25T20:10:38.310 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": "rbd", 2024-04-25T20:10:38.310 INFO:teuthology.orchestra.run.smithi039.stdout: "create_time": "2024-04-25T20:09:14.202057+0000", 2024-04-25T20:10:38.310 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-04-25T20:10:38.311 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_names": "hashpspool", 2024-04-25T20:10:38.311 INFO:teuthology.orchestra.run.smithi039.stdout: "type": 1, 2024-04-25T20:10:38.311 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 2, 2024-04-25T20:10:38.311 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": 1, 2024-04-25T20:10:38.311 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_rule": 0, 2024-04-25T20:10:38.311 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_count": 0, 2024-04-25T20:10:38.311 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_target": 0, 2024-04-25T20:10:38.311 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T20:10:38.311 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T20:10:38.311 INFO:teuthology.orchestra.run.smithi039.stdout: "object_hash": 2, 2024-04-25T20:10:38.311 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscale_mode": "off", 2024-04-25T20:10:38.311 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": 8, 2024-04-25T20:10:38.311 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num": 8, 2024-04-25T20:10:38.311 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num_target": 8, 2024-04-25T20:10:38.312 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_target": 8, 2024-04-25T20:10:38.312 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_pending": 8, 2024-04-25T20:10:38.312 INFO:teuthology.orchestra.run.smithi039.stdout: "last_pg_merge_meta": { 2024-04-25T20:10:38.312 INFO:teuthology.orchestra.run.smithi039.stdout: "source_pgid": "0.0", 2024-04-25T20:10:38.312 INFO:teuthology.orchestra.run.smithi039.stdout: "ready_epoch": 0, 2024-04-25T20:10:38.312 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_started": 0, 2024-04-25T20:10:38.312 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_clean": 0, 2024-04-25T20:10:38.312 INFO:teuthology.orchestra.run.smithi039.stdout: "source_version": "0'0", 2024-04-25T20:10:38.312 INFO:teuthology.orchestra.run.smithi039.stdout: "target_version": "0'0" 2024-04-25T20:10:38.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.312 INFO:teuthology.orchestra.run.smithi039.stdout: "last_change": "16", 2024-04-25T20:10:38.313 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend": "0", 2024-04-25T20:10:38.313 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T20:10:38.313 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T20:10:38.313 INFO:teuthology.orchestra.run.smithi039.stdout: "auid": 0, 2024-04-25T20:10:38.313 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_mode": "selfmanaged", 2024-04-25T20:10:38.313 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_seq": 0, 2024-04-25T20:10:38.313 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_epoch": 0, 2024-04-25T20:10:38.313 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_snaps": [], 2024-04-25T20:10:38.313 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps": "[]", 2024-04-25T20:10:38.313 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_bytes": 0, 2024-04-25T20:10:38.313 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_objects": 0, 2024-04-25T20:10:38.314 INFO:teuthology.orchestra.run.smithi039.stdout: "tiers": [], 2024-04-25T20:10:38.314 INFO:teuthology.orchestra.run.smithi039.stdout: "tier_of": -1, 2024-04-25T20:10:38.314 INFO:teuthology.orchestra.run.smithi039.stdout: "read_tier": -1, 2024-04-25T20:10:38.314 INFO:teuthology.orchestra.run.smithi039.stdout: "write_tier": -1, 2024-04-25T20:10:38.314 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_mode": "none", 2024-04-25T20:10:38.314 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_bytes": 0, 2024-04-25T20:10:38.314 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_objects": 0, 2024-04-25T20:10:38.314 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T20:10:38.314 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T20:10:38.314 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T20:10:38.314 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_flush_age": 0, 2024-04-25T20:10:38.314 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_evict_age": 0, 2024-04-25T20:10:38.314 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profile": "", 2024-04-25T20:10:38.315 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_params": { 2024-04-25T20:10:38.315 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "none" 2024-04-25T20:10:38.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.315 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_period": 0, 2024-04-25T20:10:38.315 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_count": 0, 2024-04-25T20:10:38.315 INFO:teuthology.orchestra.run.smithi039.stdout: "use_gmt_hitset": true, 2024-04-25T20:10:38.315 INFO:teuthology.orchestra.run.smithi039.stdout: "min_read_recency_for_promote": 0, 2024-04-25T20:10:38.315 INFO:teuthology.orchestra.run.smithi039.stdout: "min_write_recency_for_promote": 0, 2024-04-25T20:10:38.315 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T20:10:38.315 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_search_last_n": 0, 2024-04-25T20:10:38.315 INFO:teuthology.orchestra.run.smithi039.stdout: "grade_table": [], 2024-04-25T20:10:38.315 INFO:teuthology.orchestra.run.smithi039.stdout: "stripe_width": 0, 2024-04-25T20:10:38.315 INFO:teuthology.orchestra.run.smithi039.stdout: "expected_num_objects": 0, 2024-04-25T20:10:38.315 INFO:teuthology.orchestra.run.smithi039.stdout: "fast_read": false, 2024-04-25T20:10:38.316 INFO:teuthology.orchestra.run.smithi039.stdout: "options": {}, 2024-04-25T20:10:38.316 INFO:teuthology.orchestra.run.smithi039.stdout: "application_metadata": { 2024-04-25T20:10:38.316 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd": {} 2024-04-25T20:10:38.316 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.316 INFO:teuthology.orchestra.run.smithi039.stdout: "read_balance": { 2024-04-25T20:10:38.316 INFO:teuthology.orchestra.run.smithi039.stdout: "score_acting": 1.7400000095367432, 2024-04-25T20:10:38.316 INFO:teuthology.orchestra.run.smithi039.stdout: "score_stable": 1.7400000095367432, 2024-04-25T20:10:38.316 INFO:teuthology.orchestra.run.smithi039.stdout: "optimal_score": 0.86000001430511475, 2024-04-25T20:10:38.316 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_acting": 1.5, 2024-04-25T20:10:38.316 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_stable": 1.5, 2024-04-25T20:10:38.316 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity_weighted": 0.81888771057128906, 2024-04-25T20:10:38.316 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity": 0.93999999761581421, 2024-04-25T20:10:38.316 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity_weighted": 0.81999999284744263 2024-04-25T20:10:38.316 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.317 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.317 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-04-25T20:10:38.317 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": ".mgr", 2024-04-25T20:10:38.317 INFO:teuthology.orchestra.run.smithi039.stdout: "create_time": "2024-04-25T20:09:15.645688+0000", 2024-04-25T20:10:38.317 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-04-25T20:10:38.317 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_names": "hashpspool", 2024-04-25T20:10:38.317 INFO:teuthology.orchestra.run.smithi039.stdout: "type": 1, 2024-04-25T20:10:38.317 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 2, 2024-04-25T20:10:38.317 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": 1, 2024-04-25T20:10:38.317 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_rule": 0, 2024-04-25T20:10:38.317 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_count": 0, 2024-04-25T20:10:38.317 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_target": 0, 2024-04-25T20:10:38.317 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T20:10:38.318 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T20:10:38.318 INFO:teuthology.orchestra.run.smithi039.stdout: "object_hash": 2, 2024-04-25T20:10:38.318 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscale_mode": "off", 2024-04-25T20:10:38.318 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": 1, 2024-04-25T20:10:38.318 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num": 1, 2024-04-25T20:10:38.318 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num_target": 1, 2024-04-25T20:10:38.318 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_target": 1, 2024-04-25T20:10:38.318 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_pending": 1, 2024-04-25T20:10:38.318 INFO:teuthology.orchestra.run.smithi039.stdout: "last_pg_merge_meta": { 2024-04-25T20:10:38.318 INFO:teuthology.orchestra.run.smithi039.stdout: "source_pgid": "0.0", 2024-04-25T20:10:38.318 INFO:teuthology.orchestra.run.smithi039.stdout: "ready_epoch": 0, 2024-04-25T20:10:38.318 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_started": 0, 2024-04-25T20:10:38.318 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_clean": 0, 2024-04-25T20:10:38.318 INFO:teuthology.orchestra.run.smithi039.stdout: "source_version": "0'0", 2024-04-25T20:10:38.319 INFO:teuthology.orchestra.run.smithi039.stdout: "target_version": "0'0" 2024-04-25T20:10:38.319 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.319 INFO:teuthology.orchestra.run.smithi039.stdout: "last_change": "17", 2024-04-25T20:10:38.319 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend": "0", 2024-04-25T20:10:38.319 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T20:10:38.319 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T20:10:38.319 INFO:teuthology.orchestra.run.smithi039.stdout: "auid": 0, 2024-04-25T20:10:38.319 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_mode": "selfmanaged", 2024-04-25T20:10:38.319 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_seq": 0, 2024-04-25T20:10:38.319 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_epoch": 0, 2024-04-25T20:10:38.319 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_snaps": [], 2024-04-25T20:10:38.319 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps": "[]", 2024-04-25T20:10:38.319 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_bytes": 0, 2024-04-25T20:10:38.319 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_objects": 0, 2024-04-25T20:10:38.320 INFO:teuthology.orchestra.run.smithi039.stdout: "tiers": [], 2024-04-25T20:10:38.320 INFO:teuthology.orchestra.run.smithi039.stdout: "tier_of": -1, 2024-04-25T20:10:38.320 INFO:teuthology.orchestra.run.smithi039.stdout: "read_tier": -1, 2024-04-25T20:10:38.320 INFO:teuthology.orchestra.run.smithi039.stdout: "write_tier": -1, 2024-04-25T20:10:38.320 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_mode": "none", 2024-04-25T20:10:38.320 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_bytes": 0, 2024-04-25T20:10:38.320 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_objects": 0, 2024-04-25T20:10:38.320 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T20:10:38.320 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T20:10:38.320 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T20:10:38.320 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_flush_age": 0, 2024-04-25T20:10:38.320 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_evict_age": 0, 2024-04-25T20:10:38.320 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profile": "", 2024-04-25T20:10:38.321 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_params": { 2024-04-25T20:10:38.321 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "none" 2024-04-25T20:10:38.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.321 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_period": 0, 2024-04-25T20:10:38.321 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_count": 0, 2024-04-25T20:10:38.321 INFO:teuthology.orchestra.run.smithi039.stdout: "use_gmt_hitset": true, 2024-04-25T20:10:38.321 INFO:teuthology.orchestra.run.smithi039.stdout: "min_read_recency_for_promote": 0, 2024-04-25T20:10:38.321 INFO:teuthology.orchestra.run.smithi039.stdout: "min_write_recency_for_promote": 0, 2024-04-25T20:10:38.321 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T20:10:38.321 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_search_last_n": 0, 2024-04-25T20:10:38.321 INFO:teuthology.orchestra.run.smithi039.stdout: "grade_table": [], 2024-04-25T20:10:38.321 INFO:teuthology.orchestra.run.smithi039.stdout: "stripe_width": 0, 2024-04-25T20:10:38.321 INFO:teuthology.orchestra.run.smithi039.stdout: "expected_num_objects": 0, 2024-04-25T20:10:38.321 INFO:teuthology.orchestra.run.smithi039.stdout: "fast_read": false, 2024-04-25T20:10:38.321 INFO:teuthology.orchestra.run.smithi039.stdout: "options": { 2024-04-25T20:10:38.321 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_max": 32, 2024-04-25T20:10:38.322 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_min": 1 2024-04-25T20:10:38.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.322 INFO:teuthology.orchestra.run.smithi039.stdout: "application_metadata": { 2024-04-25T20:10:38.322 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": {} 2024-04-25T20:10:38.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.322 INFO:teuthology.orchestra.run.smithi039.stdout: "read_balance": { 2024-04-25T20:10:38.322 INFO:teuthology.orchestra.run.smithi039.stdout: "score_acting": 6.9000000953674316, 2024-04-25T20:10:38.322 INFO:teuthology.orchestra.run.smithi039.stdout: "score_stable": 6.9000000953674316, 2024-04-25T20:10:38.322 INFO:teuthology.orchestra.run.smithi039.stdout: "optimal_score": 0.28999999165534973, 2024-04-25T20:10:38.322 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_acting": 2, 2024-04-25T20:10:38.322 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_stable": 2, 2024-04-25T20:10:38.322 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity_weighted": 0.81888771057128906, 2024-04-25T20:10:38.322 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity": 0.93999999761581421, 2024-04-25T20:10:38.322 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity_weighted": 0.81999999284744263 2024-04-25T20:10:38.323 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.323 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.323 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-04-25T20:10:38.323 INFO:teuthology.orchestra.run.smithi039.stdout: "osds": [ 2024-04-25T20:10:38.323 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.323 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 0, 2024-04-25T20:10:38.323 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "7a7118ba-4aee-404f-97e3-94c50e2996c4", 2024-04-25T20:10:38.323 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-04-25T20:10:38.323 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-04-25T20:10:38.323 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-04-25T20:10:38.323 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-04-25T20:10:38.323 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-04-25T20:10:38.323 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-04-25T20:10:38.323 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 13, 2024-04-25T20:10:38.323 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 52, 2024-04-25T20:10:38.323 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-04-25T20:10:38.324 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-04-25T20:10:38.324 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-04-25T20:10:38.324 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.324 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.324 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.324 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6816", 2024-04-25T20:10:38.324 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31832 2024-04-25T20:10:38.324 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.324 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.324 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.324 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6818", 2024-04-25T20:10:38.324 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31832 2024-04-25T20:10:38.324 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.324 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.324 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.324 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-04-25T20:10:38.325 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.325 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.325 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.325 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6820", 2024-04-25T20:10:38.325 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31832 2024-04-25T20:10:38.325 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.325 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.325 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.325 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6822", 2024-04-25T20:10:38.325 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31832 2024-04-25T20:10:38.325 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.325 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.326 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-04-25T20:10:38.326 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.326 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.326 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.326 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6828", 2024-04-25T20:10:38.326 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31832 2024-04-25T20:10:38.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.326 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.326 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.326 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6830", 2024-04-25T20:10:38.326 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31832 2024-04-25T20:10:38.327 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.327 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.327 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-04-25T20:10:38.327 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.327 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.327 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.327 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6824", 2024-04-25T20:10:38.327 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31832 2024-04-25T20:10:38.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.327 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.327 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.328 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6826", 2024-04-25T20:10:38.328 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31832 2024-04-25T20:10:38.328 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.328 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.328 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.328 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6818/31832", 2024-04-25T20:10:38.328 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6822/31832", 2024-04-25T20:10:38.328 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6830/31832", 2024-04-25T20:10:38.328 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6826/31832", 2024-04-25T20:10:38.328 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-04-25T20:10:38.328 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-04-25T20:10:38.329 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-04-25T20:10:38.329 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.329 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.329 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 1, 2024-04-25T20:10:38.329 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "13637dbe-e37f-457f-b68b-8eed1b970617", 2024-04-25T20:10:38.329 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-04-25T20:10:38.329 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-04-25T20:10:38.329 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-04-25T20:10:38.329 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-04-25T20:10:38.329 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-04-25T20:10:38.330 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-04-25T20:10:38.330 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 13, 2024-04-25T20:10:38.330 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 14, 2024-04-25T20:10:38.330 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-04-25T20:10:38.330 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-04-25T20:10:38.330 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-04-25T20:10:38.330 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.330 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.330 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.330 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6808", 2024-04-25T20:10:38.330 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31831 2024-04-25T20:10:38.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.331 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.331 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6809", 2024-04-25T20:10:38.331 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31831 2024-04-25T20:10:38.331 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.331 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.331 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-04-25T20:10:38.331 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.331 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.331 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6810", 2024-04-25T20:10:38.332 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31831 2024-04-25T20:10:38.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.332 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.332 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6811", 2024-04-25T20:10:38.332 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31831 2024-04-25T20:10:38.332 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.332 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.332 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-04-25T20:10:38.332 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.333 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.333 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6814", 2024-04-25T20:10:38.333 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31831 2024-04-25T20:10:38.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.333 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.333 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.333 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6815", 2024-04-25T20:10:38.333 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31831 2024-04-25T20:10:38.333 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.333 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.333 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-04-25T20:10:38.333 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.334 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.334 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.334 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6812", 2024-04-25T20:10:38.334 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31831 2024-04-25T20:10:38.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.334 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.334 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.334 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6813", 2024-04-25T20:10:38.334 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31831 2024-04-25T20:10:38.334 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.334 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.335 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6809/31831", 2024-04-25T20:10:38.335 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6811/31831", 2024-04-25T20:10:38.335 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6815/31831", 2024-04-25T20:10:38.335 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6813/31831", 2024-04-25T20:10:38.335 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-04-25T20:10:38.335 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-04-25T20:10:38.335 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-04-25T20:10:38.335 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.335 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 2, 2024-04-25T20:10:38.335 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "29e20ce2-4776-455d-8cf4-d539e738a6a7", 2024-04-25T20:10:38.335 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-04-25T20:10:38.335 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 0, 2024-04-25T20:10:38.336 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 0, 2024-04-25T20:10:38.336 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-04-25T20:10:38.336 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-04-25T20:10:38.336 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-04-25T20:10:38.336 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 13, 2024-04-25T20:10:38.336 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 0, 2024-04-25T20:10:38.336 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-04-25T20:10:38.336 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-04-25T20:10:38.336 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-04-25T20:10:38.336 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.336 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.336 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.337 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6800", 2024-04-25T20:10:38.337 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31829 2024-04-25T20:10:38.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.337 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.337 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.337 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6801", 2024-04-25T20:10:38.337 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31829 2024-04-25T20:10:38.337 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.337 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.337 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-04-25T20:10:38.337 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.337 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.337 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.338 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6802", 2024-04-25T20:10:38.338 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31829 2024-04-25T20:10:38.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.338 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.338 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.338 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6803", 2024-04-25T20:10:38.338 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31829 2024-04-25T20:10:38.338 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.338 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.338 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-04-25T20:10:38.339 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.339 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.339 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.339 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6806", 2024-04-25T20:10:38.339 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31829 2024-04-25T20:10:38.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.339 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.339 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.339 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6807", 2024-04-25T20:10:38.339 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31829 2024-04-25T20:10:38.339 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.339 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.339 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-04-25T20:10:38.339 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.340 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.340 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.340 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6804", 2024-04-25T20:10:38.340 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31829 2024-04-25T20:10:38.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.340 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.340 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.340 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6805", 2024-04-25T20:10:38.340 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31829 2024-04-25T20:10:38.340 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.340 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.340 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6801/31829", 2024-04-25T20:10:38.340 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6803/31829", 2024-04-25T20:10:38.340 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6807/31829", 2024-04-25T20:10:38.341 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6805/31829", 2024-04-25T20:10:38.341 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-04-25T20:10:38.341 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-04-25T20:10:38.341 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-04-25T20:10:38.341 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.341 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.341 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 3, 2024-04-25T20:10:38.341 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "a8b34e02-0898-4303-9fd5-62cbba9fec5c", 2024-04-25T20:10:38.341 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-04-25T20:10:38.341 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 0, 2024-04-25T20:10:38.341 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 0, 2024-04-25T20:10:38.341 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-04-25T20:10:38.341 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-04-25T20:10:38.342 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-04-25T20:10:38.342 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 13, 2024-04-25T20:10:38.342 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 14, 2024-04-25T20:10:38.342 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-04-25T20:10:38.342 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-04-25T20:10:38.342 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-04-25T20:10:38.342 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.342 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.342 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6817", 2024-04-25T20:10:38.342 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31830 2024-04-25T20:10:38.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.343 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.343 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6819", 2024-04-25T20:10:38.343 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31830 2024-04-25T20:10:38.343 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.343 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.343 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-04-25T20:10:38.343 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.343 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.343 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.343 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6821", 2024-04-25T20:10:38.343 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31830 2024-04-25T20:10:38.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.344 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.344 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.344 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6823", 2024-04-25T20:10:38.344 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31830 2024-04-25T20:10:38.344 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.344 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.344 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-04-25T20:10:38.344 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.344 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.344 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.344 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6829", 2024-04-25T20:10:38.344 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31830 2024-04-25T20:10:38.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.345 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.345 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.345 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6831", 2024-04-25T20:10:38.345 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31830 2024-04-25T20:10:38.345 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.345 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.345 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-04-25T20:10:38.345 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.345 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.345 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.345 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6825", 2024-04-25T20:10:38.346 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31830 2024-04-25T20:10:38.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.346 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.346 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.346 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6827", 2024-04-25T20:10:38.346 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31830 2024-04-25T20:10:38.346 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.346 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.346 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6819/31830", 2024-04-25T20:10:38.346 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6823/31830", 2024-04-25T20:10:38.346 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6831/31830", 2024-04-25T20:10:38.346 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6827/31830", 2024-04-25T20:10:38.346 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-04-25T20:10:38.347 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-04-25T20:10:38.347 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-04-25T20:10:38.347 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.347 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 4, 2024-04-25T20:10:38.347 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "e92b5c67-6908-4fd4-9940-5cb98b5ee847", 2024-04-25T20:10:38.347 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-04-25T20:10:38.347 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-04-25T20:10:38.347 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-04-25T20:10:38.347 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 0.5724334716796875, 2024-04-25T20:10:38.347 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-04-25T20:10:38.347 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-04-25T20:10:38.347 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 13, 2024-04-25T20:10:38.347 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 34, 2024-04-25T20:10:38.348 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-04-25T20:10:38.348 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-04-25T20:10:38.348 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-04-25T20:10:38.348 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.348 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.348 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6800", 2024-04-25T20:10:38.348 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31724 2024-04-25T20:10:38.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.348 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.348 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6801", 2024-04-25T20:10:38.348 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31724 2024-04-25T20:10:38.348 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.349 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.349 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-04-25T20:10:38.349 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.349 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.349 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6802", 2024-04-25T20:10:38.349 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31724 2024-04-25T20:10:38.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.349 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.349 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6803", 2024-04-25T20:10:38.349 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31724 2024-04-25T20:10:38.349 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.350 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.350 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-04-25T20:10:38.350 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.350 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.350 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.350 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6806", 2024-04-25T20:10:38.350 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31724 2024-04-25T20:10:38.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.350 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.350 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.350 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6807", 2024-04-25T20:10:38.350 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31724 2024-04-25T20:10:38.351 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.351 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.351 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-04-25T20:10:38.351 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.351 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.351 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.351 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6804", 2024-04-25T20:10:38.351 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31724 2024-04-25T20:10:38.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.351 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.351 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.351 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6805", 2024-04-25T20:10:38.351 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31724 2024-04-25T20:10:38.351 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.351 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.352 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.135:6801/31724", 2024-04-25T20:10:38.352 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.135:6803/31724", 2024-04-25T20:10:38.352 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.135:6807/31724", 2024-04-25T20:10:38.352 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.135:6805/31724", 2024-04-25T20:10:38.352 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-04-25T20:10:38.352 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-04-25T20:10:38.352 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-04-25T20:10:38.352 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.352 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 5, 2024-04-25T20:10:38.352 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "d74d8fef-0211-413b-bff2-f97776fc1faa", 2024-04-25T20:10:38.352 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-04-25T20:10:38.352 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-04-25T20:10:38.353 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-04-25T20:10:38.353 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 0, 2024-04-25T20:10:38.353 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-04-25T20:10:38.353 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-04-25T20:10:38.353 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 13, 2024-04-25T20:10:38.353 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 14, 2024-04-25T20:10:38.353 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-04-25T20:10:38.353 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-04-25T20:10:38.353 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-04-25T20:10:38.353 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.353 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.353 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6820", 2024-04-25T20:10:38.354 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31734 2024-04-25T20:10:38.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.354 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.354 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6822", 2024-04-25T20:10:38.354 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31734 2024-04-25T20:10:38.354 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.354 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.354 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-04-25T20:10:38.354 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.354 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.357 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.357 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6824", 2024-04-25T20:10:38.357 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31734 2024-04-25T20:10:38.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.357 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.357 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.357 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6826", 2024-04-25T20:10:38.357 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31734 2024-04-25T20:10:38.357 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.358 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.358 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-04-25T20:10:38.358 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.358 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.358 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6830", 2024-04-25T20:10:38.358 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31734 2024-04-25T20:10:38.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.358 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.358 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6831", 2024-04-25T20:10:38.359 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31734 2024-04-25T20:10:38.359 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.359 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.359 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-04-25T20:10:38.359 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.359 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.359 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6828", 2024-04-25T20:10:38.359 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31734 2024-04-25T20:10:38.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.359 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.360 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6829", 2024-04-25T20:10:38.360 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31734 2024-04-25T20:10:38.360 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.360 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.360 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.135:6822/31734", 2024-04-25T20:10:38.360 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.135:6826/31734", 2024-04-25T20:10:38.360 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.135:6831/31734", 2024-04-25T20:10:38.360 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.135:6829/31734", 2024-04-25T20:10:38.360 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-04-25T20:10:38.360 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-04-25T20:10:38.360 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-04-25T20:10:38.360 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.361 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.361 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 6, 2024-04-25T20:10:38.361 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "7dc84c6e-76d8-4f22-8b17-51f62765762d", 2024-04-25T20:10:38.361 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-04-25T20:10:38.361 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-04-25T20:10:38.361 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-04-25T20:10:38.361 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 0.978668212890625, 2024-04-25T20:10:38.361 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-04-25T20:10:38.361 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-04-25T20:10:38.362 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 13, 2024-04-25T20:10:38.362 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 25, 2024-04-25T20:10:38.362 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-04-25T20:10:38.362 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-04-25T20:10:38.362 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-04-25T20:10:38.362 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.362 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.362 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6808", 2024-04-25T20:10:38.362 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31725 2024-04-25T20:10:38.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.363 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.363 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6809", 2024-04-25T20:10:38.363 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31725 2024-04-25T20:10:38.363 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.363 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.363 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-04-25T20:10:38.363 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.363 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.363 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6810", 2024-04-25T20:10:38.363 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31725 2024-04-25T20:10:38.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.363 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.364 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6811", 2024-04-25T20:10:38.364 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31725 2024-04-25T20:10:38.364 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.364 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.364 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-04-25T20:10:38.364 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.364 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.364 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.364 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6814", 2024-04-25T20:10:38.364 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31725 2024-04-25T20:10:38.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.364 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.365 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.365 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6815", 2024-04-25T20:10:38.365 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31725 2024-04-25T20:10:38.365 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.365 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.365 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-04-25T20:10:38.365 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.365 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.365 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.365 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6812", 2024-04-25T20:10:38.365 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31725 2024-04-25T20:10:38.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.365 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.366 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.366 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6813", 2024-04-25T20:10:38.366 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31725 2024-04-25T20:10:38.366 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.366 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.366 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.135:6809/31725", 2024-04-25T20:10:38.366 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.135:6811/31725", 2024-04-25T20:10:38.366 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.135:6815/31725", 2024-04-25T20:10:38.366 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.135:6813/31725", 2024-04-25T20:10:38.366 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-04-25T20:10:38.366 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-04-25T20:10:38.366 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-04-25T20:10:38.367 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.367 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 7, 2024-04-25T20:10:38.367 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "7adb3187-aed3-48af-868c-7230e9bfc85e", 2024-04-25T20:10:38.367 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-04-25T20:10:38.367 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-04-25T20:10:38.367 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 0.9459075927734375, 2024-04-25T20:10:38.367 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-04-25T20:10:38.367 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-04-25T20:10:38.367 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-04-25T20:10:38.367 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 13, 2024-04-25T20:10:38.367 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 25, 2024-04-25T20:10:38.368 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-04-25T20:10:38.368 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-04-25T20:10:38.368 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-04-25T20:10:38.368 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.368 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.368 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6816", 2024-04-25T20:10:38.368 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31737 2024-04-25T20:10:38.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.368 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.368 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6817", 2024-04-25T20:10:38.368 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31737 2024-04-25T20:10:38.369 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.369 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.369 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-04-25T20:10:38.369 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.369 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.369 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.369 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6818", 2024-04-25T20:10:38.369 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31737 2024-04-25T20:10:38.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.369 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.369 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.369 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6819", 2024-04-25T20:10:38.369 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31737 2024-04-25T20:10:38.370 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.370 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.370 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-04-25T20:10:38.370 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.370 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.370 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.370 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6825", 2024-04-25T20:10:38.370 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31737 2024-04-25T20:10:38.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.370 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.371 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.371 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6827", 2024-04-25T20:10:38.371 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31737 2024-04-25T20:10:38.371 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.371 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.371 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-04-25T20:10:38.371 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:38.371 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.371 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:38.371 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6821", 2024-04-25T20:10:38.371 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31737 2024-04-25T20:10:38.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.372 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:38.372 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6823", 2024-04-25T20:10:38.372 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31737 2024-04-25T20:10:38.372 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.372 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.372 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.135:6817/31737", 2024-04-25T20:10:38.372 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.135:6819/31737", 2024-04-25T20:10:38.372 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.135:6827/31737", 2024-04-25T20:10:38.372 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.135:6823/31737", 2024-04-25T20:10:38.372 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-04-25T20:10:38.373 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-04-25T20:10:38.373 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-04-25T20:10:38.373 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.373 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.373 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-04-25T20:10:38.373 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_xinfo": [ 2024-04-25T20:10:38.373 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.373 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 0, 2024-04-25T20:10:38.373 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-04-25T20:10:38.373 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-04-25T20:10:38.373 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-04-25T20:10:38.373 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540138322906710015, 2024-04-25T20:10:38.373 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-04-25T20:10:38.373 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-04-25T20:09:10.683465+0000", 2024-04-25T20:10:38.374 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-04-25T20:10:38.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.374 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 1, 2024-04-25T20:10:38.374 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-04-25T20:10:38.374 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-04-25T20:10:38.374 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-04-25T20:10:38.374 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540138322906710015, 2024-04-25T20:10:38.374 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-04-25T20:10:38.374 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-04-25T20:09:10.722423+0000", 2024-04-25T20:10:38.374 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-04-25T20:10:38.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.375 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 2, 2024-04-25T20:10:38.375 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-04-25T20:10:38.375 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-04-25T20:10:38.375 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-04-25T20:10:38.375 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540138322906710015, 2024-04-25T20:10:38.375 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 47607, 2024-04-25T20:10:38.375 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-04-25T20:09:10.712691+0000", 2024-04-25T20:10:38.375 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-04-25T20:10:38.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.375 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.375 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 3, 2024-04-25T20:10:38.375 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-04-25T20:10:38.375 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-04-25T20:10:38.376 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-04-25T20:10:38.376 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540138322906710015, 2024-04-25T20:10:38.376 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 65536, 2024-04-25T20:10:38.376 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-04-25T20:09:10.673892+0000", 2024-04-25T20:10:38.376 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-04-25T20:10:38.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.376 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.376 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 4, 2024-04-25T20:10:38.376 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-04-25T20:10:38.376 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-04-25T20:10:38.376 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-04-25T20:10:38.376 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540138322906710015, 2024-04-25T20:10:38.376 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-04-25T20:10:38.376 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-04-25T20:09:10.768469+0000", 2024-04-25T20:10:38.376 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-04-25T20:10:38.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.377 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.377 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 5, 2024-04-25T20:10:38.377 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-04-25T20:10:38.377 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-04-25T20:10:38.377 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-04-25T20:10:38.377 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540138322906710015, 2024-04-25T20:10:38.377 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-04-25T20:10:38.377 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-04-25T20:09:10.783897+0000", 2024-04-25T20:10:38.377 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-04-25T20:10:38.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.377 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.377 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 6, 2024-04-25T20:10:38.377 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-04-25T20:10:38.377 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-04-25T20:10:38.377 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-04-25T20:10:38.378 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540138322906710015, 2024-04-25T20:10:38.378 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-04-25T20:10:38.378 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-04-25T20:09:10.686833+0000", 2024-04-25T20:10:38.378 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-04-25T20:10:38.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.378 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.378 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 7, 2024-04-25T20:10:38.378 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-04-25T20:10:38.378 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-04-25T20:10:38.378 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-04-25T20:10:38.378 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540138322906710015, 2024-04-25T20:10:38.378 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-04-25T20:10:38.378 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-04-25T20:09:10.739675+0000", 2024-04-25T20:10:38.378 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-04-25T20:10:38.378 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.379 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-04-25T20:10:38.379 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_upmap": [], 2024-04-25T20:10:38.379 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_upmap_items": [], 2024-04-25T20:10:38.379 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_upmap_primaries": [], 2024-04-25T20:10:38.379 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_temp": [], 2024-04-25T20:10:38.379 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_temp": [], 2024-04-25T20:10:38.379 INFO:teuthology.orchestra.run.smithi039.stdout: "blocklist": {}, 2024-04-25T20:10:38.379 INFO:teuthology.orchestra.run.smithi039.stdout: "range_blocklist": {}, 2024-04-25T20:10:38.379 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profiles": { 2024-04-25T20:10:38.379 INFO:teuthology.orchestra.run.smithi039.stdout: "default": { 2024-04-25T20:10:38.379 INFO:teuthology.orchestra.run.smithi039.stdout: "crush-failure-domain": "osd", 2024-04-25T20:10:38.379 INFO:teuthology.orchestra.run.smithi039.stdout: "k": "2", 2024-04-25T20:10:38.379 INFO:teuthology.orchestra.run.smithi039.stdout: "m": "1", 2024-04-25T20:10:38.379 INFO:teuthology.orchestra.run.smithi039.stdout: "plugin": "jerasure", 2024-04-25T20:10:38.380 INFO:teuthology.orchestra.run.smithi039.stdout: "technique": "reed_sol_van" 2024-04-25T20:10:38.380 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.380 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps_queue": [], 2024-04-25T20:10:38.380 INFO:teuthology.orchestra.run.smithi039.stdout: "new_removed_snaps": [], 2024-04-25T20:10:38.380 INFO:teuthology.orchestra.run.smithi039.stdout: "new_purged_snaps": [], 2024-04-25T20:10:38.380 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_node_flags": {}, 2024-04-25T20:10:38.380 INFO:teuthology.orchestra.run.smithi039.stdout: "device_class_flags": {}, 2024-04-25T20:10:38.380 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_mode": { 2024-04-25T20:10:38.380 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_mode_enabled": false, 2024-04-25T20:10:38.380 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_bucket_count": 0, 2024-04-25T20:10:38.381 INFO:teuthology.orchestra.run.smithi039.stdout: "degraded_stretch_mode": 0, 2024-04-25T20:10:38.381 INFO:teuthology.orchestra.run.smithi039.stdout: "recovering_stretch_mode": 0, 2024-04-25T20:10:38.381 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_mode_bucket": 0 2024-04-25T20:10:38.381 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.381 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:38.381 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:38.381 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:38.382 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:38.382 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:38.383 INFO:tasks.thrashosds.thrasher:j is {'epoch': 54, 'fsid': '975ceafd-555d-4f71-81e4-1f70364da00f', 'created': '2024-04-25T20:09:03.377320+0000', 'modified': '2024-04-25T20:10:36.840457+0000', 'last_up_change': '2024-04-25T20:09:13.740341+0000', 'last_in_change': '2024-04-25T20:10:08.354345+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-04-25T20:09:14.202057+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8600000143051147, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.8188877105712891, 'average_primary_affinity': 0.9399999976158142, 'average_primary_affinity_weighted': 0.8199999928474426}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-04-25T20:09:15.645688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8188877105712891, 'average_primary_affinity': 0.9399999976158142, 'average_primary_affinity_weighted': 0.8199999928474426}}], 'osds': [{'osd': 0, 'uuid': '7a7118ba-4aee-404f-97e3-94c50e2996c4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 52, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6816', 'nonce': 31832}, {'type': 'v1', 'addr': '172.21.15.39:6818', 'nonce': 31832}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6820', 'nonce': 31832}, {'type': 'v1', 'addr': '172.21.15.39:6822', 'nonce': 31832}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6828', 'nonce': 31832}, {'type': 'v1', 'addr': '172.21.15.39:6830', 'nonce': 31832}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6824', 'nonce': 31832}, {'type': 'v1', 'addr': '172.21.15.39:6826', 'nonce': 31832}]}, 'public_addr': '172.21.15.39:6818/31832', 'cluster_addr': '172.21.15.39:6822/31832', 'heartbeat_back_addr': '172.21.15.39:6830/31832', 'heartbeat_front_addr': '172.21.15.39:6826/31832', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '13637dbe-e37f-457f-b68b-8eed1b970617', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 14, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6808', 'nonce': 31831}, {'type': 'v1', 'addr': '172.21.15.39:6809', 'nonce': 31831}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6810', 'nonce': 31831}, {'type': 'v1', 'addr': '172.21.15.39:6811', 'nonce': 31831}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6814', 'nonce': 31831}, {'type': 'v1', 'addr': '172.21.15.39:6815', 'nonce': 31831}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6812', 'nonce': 31831}, {'type': 'v1', 'addr': '172.21.15.39:6813', 'nonce': 31831}]}, 'public_addr': '172.21.15.39:6809/31831', 'cluster_addr': '172.21.15.39:6811/31831', 'heartbeat_back_addr': '172.21.15.39:6815/31831', 'heartbeat_front_addr': '172.21.15.39:6813/31831', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '29e20ce2-4776-455d-8cf4-d539e738a6a7', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6800', 'nonce': 31829}, {'type': 'v1', 'addr': '172.21.15.39:6801', 'nonce': 31829}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6802', 'nonce': 31829}, {'type': 'v1', 'addr': '172.21.15.39:6803', 'nonce': 31829}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6806', 'nonce': 31829}, {'type': 'v1', 'addr': '172.21.15.39:6807', 'nonce': 31829}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6804', 'nonce': 31829}, {'type': 'v1', 'addr': '172.21.15.39:6805', 'nonce': 31829}]}, 'public_addr': '172.21.15.39:6801/31829', 'cluster_addr': '172.21.15.39:6803/31829', 'heartbeat_back_addr': '172.21.15.39:6807/31829', 'heartbeat_front_addr': '172.21.15.39:6805/31829', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'a8b34e02-0898-4303-9fd5-62cbba9fec5c', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 14, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6817', 'nonce': 31830}, {'type': 'v1', 'addr': '172.21.15.39:6819', 'nonce': 31830}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6821', 'nonce': 31830}, {'type': 'v1', 'addr': '172.21.15.39:6823', 'nonce': 31830}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6829', 'nonce': 31830}, {'type': 'v1', 'addr': '172.21.15.39:6831', 'nonce': 31830}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6825', 'nonce': 31830}, {'type': 'v1', 'addr': '172.21.15.39:6827', 'nonce': 31830}]}, 'public_addr': '172.21.15.39:6819/31830', 'cluster_addr': '172.21.15.39:6823/31830', 'heartbeat_back_addr': '172.21.15.39:6831/31830', 'heartbeat_front_addr': '172.21.15.39:6827/31830', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'e92b5c67-6908-4fd4-9940-5cb98b5ee847', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.5724334716796875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6800', 'nonce': 31724}, {'type': 'v1', 'addr': '172.21.15.135:6801', 'nonce': 31724}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6802', 'nonce': 31724}, {'type': 'v1', 'addr': '172.21.15.135:6803', 'nonce': 31724}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6806', 'nonce': 31724}, {'type': 'v1', 'addr': '172.21.15.135:6807', 'nonce': 31724}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6804', 'nonce': 31724}, {'type': 'v1', 'addr': '172.21.15.135:6805', 'nonce': 31724}]}, 'public_addr': '172.21.15.135:6801/31724', 'cluster_addr': '172.21.15.135:6803/31724', 'heartbeat_back_addr': '172.21.15.135:6807/31724', 'heartbeat_front_addr': '172.21.15.135:6805/31724', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'd74d8fef-0211-413b-bff2-f97776fc1faa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 14, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6820', 'nonce': 31734}, {'type': 'v1', 'addr': '172.21.15.135:6822', 'nonce': 31734}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6824', 'nonce': 31734}, {'type': 'v1', 'addr': '172.21.15.135:6826', 'nonce': 31734}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6830', 'nonce': 31734}, {'type': 'v1', 'addr': '172.21.15.135:6831', 'nonce': 31734}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6828', 'nonce': 31734}, {'type': 'v1', 'addr': '172.21.15.135:6829', 'nonce': 31734}]}, 'public_addr': '172.21.15.135:6822/31734', 'cluster_addr': '172.21.15.135:6826/31734', 'heartbeat_back_addr': '172.21.15.135:6831/31734', 'heartbeat_front_addr': '172.21.15.135:6829/31734', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '7dc84c6e-76d8-4f22-8b17-51f62765762d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.978668212890625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6808', 'nonce': 31725}, {'type': 'v1', 'addr': '172.21.15.135:6809', 'nonce': 31725}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6810', 'nonce': 31725}, {'type': 'v1', 'addr': '172.21.15.135:6811', 'nonce': 31725}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6814', 'nonce': 31725}, {'type': 'v1', 'addr': '172.21.15.135:6815', 'nonce': 31725}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6812', 'nonce': 31725}, {'type': 'v1', 'addr': '172.21.15.135:6813', 'nonce': 31725}]}, 'public_addr': '172.21.15.135:6809/31725', 'cluster_addr': '172.21.15.135:6811/31725', 'heartbeat_back_addr': '172.21.15.135:6815/31725', 'heartbeat_front_addr': '172.21.15.135:6813/31725', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '7adb3187-aed3-48af-868c-7230e9bfc85e', 'up': 1, 'in': 1, 'weight': 0.9459075927734375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6816', 'nonce': 31737}, {'type': 'v1', 'addr': '172.21.15.135:6817', 'nonce': 31737}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6818', 'nonce': 31737}, {'type': 'v1', 'addr': '172.21.15.135:6819', 'nonce': 31737}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6825', 'nonce': 31737}, {'type': 'v1', 'addr': '172.21.15.135:6827', 'nonce': 31737}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6821', 'nonce': 31737}, {'type': 'v1', 'addr': '172.21.15.135:6823', 'nonce': 31737}]}, 'public_addr': '172.21.15.135:6817/31737', 'cluster_addr': '172.21.15.135:6819/31737', 'heartbeat_back_addr': '172.21.15.135:6827/31737', 'heartbeat_front_addr': '172.21.15.135:6823/31737', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T20:09:10.683465+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T20:09:10.722423+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 47607, 'last_purged_snaps_scrub': '2024-04-25T20:09:10.712691+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-04-25T20:09:10.673892+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T20:09:10.768469+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T20:09:10.783897+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T20:09:10.686833+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T20:09:10.739675+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-04-25T20:10:38.383 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T20:10:38.384 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T20:10:38.385 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:10:38.385 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:10:38.385 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:10:38.385 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:10:38.401 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:10:38.401 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:10:38.406 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:38.405+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:38.507 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:38.506+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:38.565 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:38.565 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:38.566 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:38.566 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:38.566 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:38.576 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T20:10:38.607 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:38.607+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:38.686 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:38.686 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:38.686 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:38.687 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-04-25T20:10:38.687 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.687 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:30 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'16, mlcod=16'14)", 2024-04-25T20:10:38.687 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:10:38.687 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 81.517352817000003, 2024-04-25T20:10:38.687 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0054087670000000001, 2024-04-25T20:10:38.687 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:38.687 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:38.687 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:38.687 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.687 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:38.687 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:10:38.687 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.688 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.688 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:38.688 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:10:38.688 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.688 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.688 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:38.688 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167142+0000", 2024-04-25T20:10:38.688 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999976 2024-04-25T20:10:38.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.688 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.688 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:38.688 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167162+0000", 2024-04-25T20:10:38.688 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0109e-05 2024-04-25T20:10:38.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.689 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.689 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:38.689 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167188+0000", 2024-04-25T20:10:38.689 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6146e-05 2024-04-25T20:10:38.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.689 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.689 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:38.689 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167199+0000", 2024-04-25T20:10:38.689 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0767e-05 2024-04-25T20:10:38.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.689 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.689 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:38.689 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167256+0000", 2024-04-25T20:10:38.689 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7392000000000002e-05 2024-04-25T20:10:38.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.690 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.690 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:38.690 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167283+0000", 2024-04-25T20:10:38.690 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6577e-05 2024-04-25T20:10:38.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.690 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.690 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:38.690 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.172510+0000", 2024-04-25T20:10:38.690 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0052276759999999997 2024-04-25T20:10:38.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.690 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.690 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:38.690 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.172553+0000", 2024-04-25T20:10:38.690 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2444999999999998e-05 2024-04-25T20:10:38.690 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.691 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.691 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.691 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.691 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.691 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:31 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-04-25T20:10:38.691 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:38.691 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 81.516736738999995, 2024-04-25T20:10:38.691 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0055055479999999999, 2024-04-25T20:10:38.691 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:38.691 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:38.691 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:38.691 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.691 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:38.691 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:38.691 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.692 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.692 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:38.692 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:38.692 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.692 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.692 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:38.692 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:38.692 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:38.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.693 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.693 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:38.693 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167766+0000", 2024-04-25T20:10:38.693 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.122e-06 2024-04-25T20:10:38.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.693 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.693 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:38.693 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167776+0000", 2024-04-25T20:10:38.693 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0515e-05 2024-04-25T20:10:38.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.693 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.693 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:38.693 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167782+0000", 2024-04-25T20:10:38.693 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1149999999999999e-06 2024-04-25T20:10:38.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.694 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.694 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:38.694 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167837+0000", 2024-04-25T20:10:38.694 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4763999999999999e-05 2024-04-25T20:10:38.694 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.694 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.694 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:38.694 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167855+0000", 2024-04-25T20:10:38.694 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7589e-05 2024-04-25T20:10:38.694 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.694 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.694 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:38.694 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173221+0000", 2024-04-25T20:10:38.695 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.005365903 2024-04-25T20:10:38.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.695 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.695 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:38.695 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173266+0000", 2024-04-25T20:10:38.695 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5059999999999999e-05 2024-04-25T20:10:38.695 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.695 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.695 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.695 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.695 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:32 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-04-25T20:10:38.695 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:38.695 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 81.516361554, 2024-04-25T20:10:38.695 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0051558239999999998, 2024-04-25T20:10:38.696 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:38.696 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:38.696 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:38.696 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.696 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:38.696 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:38.696 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.696 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.696 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:38.696 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:38.697 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.697 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.697 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:38.697 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:38.697 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:38.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.697 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.697 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:38.697 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168173+0000", 2024-04-25T20:10:38.697 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7663999999999997e-05 2024-04-25T20:10:38.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.697 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.697 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:38.697 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168182+0000", 2024-04-25T20:10:38.698 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0650000000000005e-06 2024-04-25T20:10:38.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.698 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.698 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:38.698 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168188+0000", 2024-04-25T20:10:38.698 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0589999999999999e-06 2024-04-25T20:10:38.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.698 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.698 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:38.698 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168239+0000", 2024-04-25T20:10:38.699 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1467e-05 2024-04-25T20:10:38.699 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.699 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.699 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:38.699 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168260+0000", 2024-04-25T20:10:38.699 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0531e-05 2024-04-25T20:10:38.699 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.699 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.699 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:38.699 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173268+0000", 2024-04-25T20:10:38.699 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0050087980000000001 2024-04-25T20:10:38.699 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.699 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.699 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:38.700 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173291+0000", 2024-04-25T20:10:38.700 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.268e-05 2024-04-25T20:10:38.700 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.700 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.700 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.700 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.700 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:33 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'19, mlcod=16'14)", 2024-04-25T20:10:38.700 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:10:38.700 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 81.515851135000005, 2024-04-25T20:10:38.700 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046697049999999997, 2024-04-25T20:10:38.700 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:38.700 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:38.700 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:38.700 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.701 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:38.701 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:10:38.701 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.701 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.701 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:38.701 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:10:38.701 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.701 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.701 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:38.701 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168645+0000", 2024-04-25T20:10:38.701 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:38.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.702 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.702 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:38.702 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168651+0000", 2024-04-25T20:10:38.702 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1079999999999998e-06 2024-04-25T20:10:38.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.702 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.702 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:38.702 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168660+0000", 2024-04-25T20:10:38.702 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0010000000000007e-06 2024-04-25T20:10:38.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.703 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.703 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:38.703 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168666+0000", 2024-04-25T20:10:38.703 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7780000000000004e-06 2024-04-25T20:10:38.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.703 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.703 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:38.703 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168701+0000", 2024-04-25T20:10:38.703 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4490999999999999e-05 2024-04-25T20:10:38.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.704 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.704 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:38.704 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168729+0000", 2024-04-25T20:10:38.704 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8476999999999999e-05 2024-04-25T20:10:38.704 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.704 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.704 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:38.704 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173294+0000", 2024-04-25T20:10:38.704 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0045650730000000002 2024-04-25T20:10:38.704 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.704 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.704 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:38.704 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173315+0000", 2024-04-25T20:10:38.704 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1245000000000001e-05 2024-04-25T20:10:38.704 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.705 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.705 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.705 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.705 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:34 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'20, mlcod=16'14)", 2024-04-25T20:10:38.705 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:10:38.705 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 81.515352206000003, 2024-04-25T20:10:38.705 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041943309999999999, 2024-04-25T20:10:38.705 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:38.705 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:38.705 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:38.705 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.705 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:38.705 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:10:38.705 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.706 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:38.706 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:10:38.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.706 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:38.706 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169144+0000", 2024-04-25T20:10:38.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:38.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.706 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:38.706 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169150+0000", 2024-04-25T20:10:38.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9150000000000001e-06 2024-04-25T20:10:38.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:38.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169159+0000", 2024-04-25T20:10:38.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8899999999999996e-06 2024-04-25T20:10:38.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:38.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169169+0000", 2024-04-25T20:10:38.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0441e-05 2024-04-25T20:10:38.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:38.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169206+0000", 2024-04-25T20:10:38.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6816000000000001e-05 2024-04-25T20:10:38.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.708 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:38.708 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169234+0000", 2024-04-25T20:10:38.708 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8008999999999999e-05 2024-04-25T20:10:38.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.708 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:38.708 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173317+0000", 2024-04-25T20:10:38.708 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0040830160000000001 2024-04-25T20:10:38.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.708 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:38.708 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173339+0000", 2024-04-25T20:10:38.708 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1897000000000001e-05 2024-04-25T20:10:38.709 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.709 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.709 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.709 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:35 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'14)", 2024-04-25T20:10:38.709 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:10:38.709 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 81.514859887, 2024-04-25T20:10:38.709 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037236529999999999, 2024-04-25T20:10:38.709 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:38.709 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:38.709 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:38.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.709 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:38.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:10:38.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:38.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:10:38.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:38.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169636+0000", 2024-04-25T20:10:38.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:38.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.711 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:38.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169678+0000", 2024-04-25T20:10:38.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1752000000000003e-05 2024-04-25T20:10:38.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.711 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:38.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169688+0000", 2024-04-25T20:10:38.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0098000000000001e-05 2024-04-25T20:10:38.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.711 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:38.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169694+0000", 2024-04-25T20:10:38.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2349999999999996e-06 2024-04-25T20:10:38.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.712 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.712 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:38.712 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169733+0000", 2024-04-25T20:10:38.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8481e-05 2024-04-25T20:10:38.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.712 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.712 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:38.712 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169750+0000", 2024-04-25T20:10:38.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7509000000000001e-05 2024-04-25T20:10:38.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.713 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:38.713 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173340+0000", 2024-04-25T20:10:38.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0035898449999999999 2024-04-25T20:10:38.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.713 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:38.713 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173361+0000", 2024-04-25T20:10:38.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.039e-05 2024-04-25T20:10:38.714 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.714 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.714 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.714 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:36 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'22, mlcod=16'14)", 2024-04-25T20:10:38.714 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:38.714 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 81.514359827000007, 2024-04-25T20:10:38.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032472120000000002, 2024-04-25T20:10:38.714 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:38.714 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:38.714 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:38.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:38.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:38.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:38.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:38.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.715 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:38.715 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:38.715 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:38.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:38.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170152+0000", 2024-04-25T20:10:38.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5798e-05 2024-04-25T20:10:38.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:38.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170161+0000", 2024-04-25T20:10:38.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.064e-06 2024-04-25T20:10:38.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.716 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.716 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:38.716 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170167+0000", 2024-04-25T20:10:38.716 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8390000000000003e-06 2024-04-25T20:10:38.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.717 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.717 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:38.717 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170202+0000", 2024-04-25T20:10:38.717 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.5120999999999998e-05 2024-04-25T20:10:38.717 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.717 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.717 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:38.717 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170219+0000", 2024-04-25T20:10:38.717 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6929999999999999e-05 2024-04-25T20:10:38.717 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.717 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.717 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:38.717 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173362+0000", 2024-04-25T20:10:38.717 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0031427619999999999 2024-04-25T20:10:38.717 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.718 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.718 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:38.718 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173384+0000", 2024-04-25T20:10:38.718 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2237000000000001e-05 2024-04-25T20:10:38.718 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.718 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.718 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.718 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.718 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:40 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'26, mlcod=16'24)", 2024-04-25T20:10:38.718 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:10:38.718 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 81.507321915000006, 2024-04-25T20:10:38.718 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028727330000000001, 2024-04-25T20:10:38.719 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:38.719 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:38.719 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:38.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:38.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:10:38.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:38.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177175+0000", 2024-04-25T20:10:38.719 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.719 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:38.719 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177173+0000", 2024-04-25T20:10:38.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999976 2024-04-25T20:10:38.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:38.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177186+0000", 2024-04-25T20:10:38.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3533000000000001e-05 2024-04-25T20:10:38.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.720 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:38.720 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177209+0000", 2024-04-25T20:10:38.720 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3280000000000001e-05 2024-04-25T20:10:38.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.720 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.721 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:38.721 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177220+0000", 2024-04-25T20:10:38.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0465000000000001e-05 2024-04-25T20:10:38.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.721 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.721 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:38.721 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177264+0000", 2024-04-25T20:10:38.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4413999999999999e-05 2024-04-25T20:10:38.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.721 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.721 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:38.721 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.177289+0000", 2024-04-25T20:10:38.721 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4783999999999999e-05 2024-04-25T20:10:38.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.722 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.722 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:38.722 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.179990+0000", 2024-04-25T20:10:38.722 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027012939999999999 2024-04-25T20:10:38.722 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.722 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.722 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:38.722 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.180048+0000", 2024-04-25T20:10:38.722 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7299000000000001e-05 2024-04-25T20:10:38.722 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.722 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.722 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.722 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.722 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.723 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:67 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'41, mlcod=16'39)", 2024-04-25T20:10:38.723 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:10:38.723 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 81.472560423000004, 2024-04-25T20:10:38.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064192549999999996, 2024-04-25T20:10:38.723 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:38.723 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:38.723 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:38.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:38.723 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:10:38.723 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.723 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:38.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:10:38.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:38.724 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211935+0000", 2024-04-25T20:10:38.724 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999981 2024-04-25T20:10:38.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.724 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.724 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:38.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211948+0000", 2024-04-25T20:10:38.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3531e-05 2024-04-25T20:10:38.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:38.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211967+0000", 2024-04-25T20:10:38.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8326999999999999e-05 2024-04-25T20:10:38.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:38.725 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211975+0000", 2024-04-25T20:10:38.725 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8589999999999994e-06 2024-04-25T20:10:38.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.725 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:38.726 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212020+0000", 2024-04-25T20:10:38.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4626999999999998e-05 2024-04-25T20:10:38.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.726 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:38.726 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212045+0000", 2024-04-25T20:10:38.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4508999999999999e-05 2024-04-25T20:10:38.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.726 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:38.726 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218316+0000", 2024-04-25T20:10:38.726 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0062711640000000001 2024-04-25T20:10:38.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.727 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:38.727 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218356+0000", 2024-04-25T20:10:38.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9993000000000001e-05 2024-04-25T20:10:38.727 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.727 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.727 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.727 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:68 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-04-25T20:10:38.727 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:38.727 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 81.472111408999993, 2024-04-25T20:10:38.727 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011231365, 2024-04-25T20:10:38.727 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:38.727 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:38.727 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:38.727 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:38.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:38.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:38.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:38.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.728 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:38.728 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:38.728 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:38.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.728 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:38.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212392+0000", 2024-04-25T20:10:38.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.79e-06 2024-04-25T20:10:38.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:38.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212406+0000", 2024-04-25T20:10:38.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3577000000000001e-05 2024-04-25T20:10:38.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.729 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.729 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:38.729 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212414+0000", 2024-04-25T20:10:38.729 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6510000000000001e-06 2024-04-25T20:10:38.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.730 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.730 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:38.730 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217114+0000", 2024-04-25T20:10:38.730 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047005379999999998 2024-04-25T20:10:38.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.730 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.730 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:38.730 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217141+0000", 2024-04-25T20:10:38.730 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6984000000000001e-05 2024-04-25T20:10:38.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.730 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.730 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:38.730 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223563+0000", 2024-04-25T20:10:38.731 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064216999999999998 2024-04-25T20:10:38.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.731 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.731 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:38.731 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223617+0000", 2024-04-25T20:10:38.731 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4011000000000002e-05 2024-04-25T20:10:38.731 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.731 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.731 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.731 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.731 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:69 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-04-25T20:10:38.731 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:10:38.731 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 81.471579219000006, 2024-04-25T20:10:38.731 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.010730094000000001, 2024-04-25T20:10:38.732 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:38.732 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:38.732 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:38.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.732 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:38.732 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:10:38.732 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.732 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:38.732 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:10:38.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:38.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212917+0000", 2024-04-25T20:10:38.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:38.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:38.733 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212957+0000", 2024-04-25T20:10:38.733 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0692000000000003e-05 2024-04-25T20:10:38.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.733 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:38.734 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212973+0000", 2024-04-25T20:10:38.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.575e-05 2024-04-25T20:10:38.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:38.734 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212980+0000", 2024-04-25T20:10:38.734 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1679999999999998e-06 2024-04-25T20:10:38.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.734 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.734 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:38.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217449+0000", 2024-04-25T20:10:38.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.004468367 2024-04-25T20:10:38.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:38.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217470+0000", 2024-04-25T20:10:38.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0749000000000001e-05 2024-04-25T20:10:38.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:38.735 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223621+0000", 2024-04-25T20:10:38.735 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0061512880000000004 2024-04-25T20:10:38.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.735 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.735 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:38.736 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223648+0000", 2024-04-25T20:10:38.736 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6961000000000002e-05 2024-04-25T20:10:38.736 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.736 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.736 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.736 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:70 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'44, mlcod=16'39)", 2024-04-25T20:10:38.736 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:38.736 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 81.470820490999998, 2024-04-25T20:10:38.736 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.013553358, 2024-04-25T20:10:38.736 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:38.736 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:38.736 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:38.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:38.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:38.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:38.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:38.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.737 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:38.737 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:38.737 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:38.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.737 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:38.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213682+0000", 2024-04-25T20:10:38.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3439999999999997e-06 2024-04-25T20:10:38.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.738 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:38.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213693+0000", 2024-04-25T20:10:38.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1049999999999999e-05 2024-04-25T20:10:38.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.738 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.738 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:38.738 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213699+0000", 2024-04-25T20:10:38.738 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9399999999999999e-06 2024-04-25T20:10:38.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.739 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:38.739 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218086+0000", 2024-04-25T20:10:38.739 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0043871240000000001 2024-04-25T20:10:38.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.739 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:38.739 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218106+0000", 2024-04-25T20:10:38.739 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.997e-05 2024-04-25T20:10:38.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.739 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.739 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:38.739 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.227186+0000", 2024-04-25T20:10:38.739 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.009079574 2024-04-25T20:10:38.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.740 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.740 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:38.740 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.227230+0000", 2024-04-25T20:10:38.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4147999999999998e-05 2024-04-25T20:10:38.740 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.740 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.740 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.740 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.740 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:106 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-04-25T20:10:38.740 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:10:38.740 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 81.099681391000004, 2024-04-25T20:10:38.740 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002936756, 2024-04-25T20:10:38.740 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:38.741 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:38.741 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:38.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.741 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:38.741 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:10:38.741 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.741 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:38.741 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:10:38.741 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.741 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:38.741 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584815+0000", 2024-04-25T20:10:38.741 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:38.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:38.742 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584860+0000", 2024-04-25T20:10:38.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5161999999999999e-05 2024-04-25T20:10:38.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:38.742 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584874+0000", 2024-04-25T20:10:38.742 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4464000000000001e-05 2024-04-25T20:10:38.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.742 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.742 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:38.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584881+0000", 2024-04-25T20:10:38.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.409e-06 2024-04-25T20:10:38.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.743 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.743 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:38.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584915+0000", 2024-04-25T20:10:38.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4007000000000002e-05 2024-04-25T20:10:38.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.743 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.743 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:38.743 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584933+0000", 2024-04-25T20:10:38.743 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7737999999999999e-05 2024-04-25T20:10:38.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.744 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.744 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:38.744 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587717+0000", 2024-04-25T20:10:38.744 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002784711 2024-04-25T20:10:38.744 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.744 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.744 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:38.744 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587752+0000", 2024-04-25T20:10:38.744 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4915e-05 2024-04-25T20:10:38.744 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.744 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.744 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.745 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:107 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'69, mlcod=16'64)", 2024-04-25T20:10:38.745 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:10:38.745 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 81.099606089000005, 2024-04-25T20:10:38.745 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028906270000000002, 2024-04-25T20:10:38.745 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:38.745 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:38.745 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:38.745 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.745 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:38.745 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:10:38.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:38.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584891+0000", 2024-04-25T20:10:38.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:38.746 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584890+0000", 2024-04-25T20:10:38.746 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:38.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.746 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.746 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:38.747 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584895+0000", 2024-04-25T20:10:38.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2069999999999997e-06 2024-04-25T20:10:38.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.747 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:38.747 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584906+0000", 2024-04-25T20:10:38.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.117e-05 2024-04-25T20:10:38.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.747 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:38.747 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584910+0000", 2024-04-25T20:10:38.747 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4190000000000004e-06 2024-04-25T20:10:38.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.748 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.748 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:38.748 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585310+0000", 2024-04-25T20:10:38.748 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00040002700000000001 2024-04-25T20:10:38.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.748 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.748 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:38.748 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585329+0000", 2024-04-25T20:10:38.748 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8838000000000002e-05 2024-04-25T20:10:38.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.748 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.748 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:38.749 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587754+0000", 2024-04-25T20:10:38.749 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0024252689999999999 2024-04-25T20:10:38.749 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.749 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.749 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:38.749 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587781+0000", 2024-04-25T20:10:38.749 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7206999999999999e-05 2024-04-25T20:10:38.749 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.749 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.749 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.749 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.749 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.750 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:108 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'70, mlcod=16'64)", 2024-04-25T20:10:38.750 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:10:38.750 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 81.099580938000003, 2024-04-25T20:10:38.750 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0028924879999999999, 2024-04-25T20:10:38.750 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:38.750 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:38.750 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:38.750 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.750 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:38.750 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:10:38.750 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.750 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.751 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:38.751 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:10:38.751 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.751 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:38.751 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584916+0000", 2024-04-25T20:10:38.751 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:38.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.751 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:38.751 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584919+0000", 2024-04-25T20:10:38.751 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.557e-06 2024-04-25T20:10:38.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.752 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:38.752 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584925+0000", 2024-04-25T20:10:38.752 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8619999999999999e-06 2024-04-25T20:10:38.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.752 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.752 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:38.752 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584931+0000", 2024-04-25T20:10:38.752 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2539999999999998e-06 2024-04-25T20:10:38.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.752 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.752 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:38.752 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585550+0000", 2024-04-25T20:10:38.752 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00061821800000000002 2024-04-25T20:10:38.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.752 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.753 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:38.753 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.585567+0000", 2024-04-25T20:10:38.753 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7499e-05 2024-04-25T20:10:38.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.753 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.753 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:38.753 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587783+0000", 2024-04-25T20:10:38.753 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0022156200000000002 2024-04-25T20:10:38.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.753 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.753 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:38.753 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587808+0000", 2024-04-25T20:10:38.753 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5729e-05 2024-04-25T20:10:38.753 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.753 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.754 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.754 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.754 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:112 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'74, mlcod=16'66)", 2024-04-25T20:10:38.754 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:10:38.754 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 81.097423316000004, 2024-04-25T20:10:38.754 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00318724, 2024-04-25T20:10:38.754 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:38.754 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:38.754 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:38.754 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.754 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:38.754 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:10:38.754 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.755 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.755 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:38.755 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:10:38.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.755 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.755 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:38.755 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587073+0000", 2024-04-25T20:10:38.755 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:38.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:38.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587136+0000", 2024-04-25T20:10:38.756 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2928999999999993e-05 2024-04-25T20:10:38.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:38.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587147+0000", 2024-04-25T20:10:38.756 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1182999999999999e-05 2024-04-25T20:10:38.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.756 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.756 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:38.756 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587153+0000", 2024-04-25T20:10:38.756 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9059999999999999e-06 2024-04-25T20:10:38.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.757 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.757 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:38.757 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587306+0000", 2024-04-25T20:10:38.757 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000152848 2024-04-25T20:10:38.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.757 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.757 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:38.757 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587330+0000", 2024-04-25T20:10:38.757 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4304e-05 2024-04-25T20:10:38.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.757 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.757 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:38.757 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.590230+0000", 2024-04-25T20:10:38.758 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029004629999999998 2024-04-25T20:10:38.758 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.758 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.758 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:38.758 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.590261+0000", 2024-04-25T20:10:38.758 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0323000000000001e-05 2024-04-25T20:10:38.758 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.758 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.758 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.758 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.758 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.758 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:118 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'80, mlcod=16'78)", 2024-04-25T20:10:38.758 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:10:38.758 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 81.092636737000007, 2024-04-25T20:10:38.759 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029618959999999999, 2024-04-25T20:10:38.759 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:38.759 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:38.759 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:38.759 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.759 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:38.759 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:10:38.759 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.759 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.759 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.759 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:38.759 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:10:38.759 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.759 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.760 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.760 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:38.760 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591859+0000", 2024-04-25T20:10:38.760 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:38.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.760 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.760 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:38.760 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591918+0000", 2024-04-25T20:10:38.760 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8842999999999999e-05 2024-04-25T20:10:38.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.760 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.760 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:38.760 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591933+0000", 2024-04-25T20:10:38.760 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4600000000000001e-05 2024-04-25T20:10:38.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.761 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:38.761 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591939+0000", 2024-04-25T20:10:38.761 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4559999999999997e-06 2024-04-25T20:10:38.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.761 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:38.761 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591990+0000", 2024-04-25T20:10:38.761 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0318999999999997e-05 2024-04-25T20:10:38.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.761 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:38.761 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.592008+0000", 2024-04-25T20:10:38.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8819000000000001e-05 2024-04-25T20:10:38.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.762 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.762 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:38.762 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.594784+0000", 2024-04-25T20:10:38.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027750850000000001 2024-04-25T20:10:38.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.762 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.762 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:38.762 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.594822+0000", 2024-04-25T20:10:38.762 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8522000000000001e-05 2024-04-25T20:10:38.762 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.762 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.762 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.763 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.763 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:139 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'97, mlcod=17'87)", 2024-04-25T20:10:38.763 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:10:38.763 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 80.558600131999995, 2024-04-25T20:10:38.763 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048546290000000001, 2024-04-25T20:10:38.763 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:38.763 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:38.763 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:38.763 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.763 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:38.764 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:10:38.764 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.764 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.764 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:38.764 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:10:38.764 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.765 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.765 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.765 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:38.765 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125896+0000", 2024-04-25T20:10:38.765 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:38.765 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.765 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.765 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:38.765 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125908+0000", 2024-04-25T20:10:38.766 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.137e-05 2024-04-25T20:10:38.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.766 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.766 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:38.766 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125915+0000", 2024-04-25T20:10:38.766 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3939999999999998e-06 2024-04-25T20:10:38.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.766 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.766 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:38.766 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125922+0000", 2024-04-25T20:10:38.766 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1930000000000004e-06 2024-04-25T20:10:38.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.767 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.767 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:38.767 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125959+0000", 2024-04-25T20:10:38.767 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6701e-05 2024-04-25T20:10:38.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.767 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.767 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:38.767 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125972+0000", 2024-04-25T20:10:38.767 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2578000000000001e-05 2024-04-25T20:10:38.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.767 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.767 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:38.767 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130719+0000", 2024-04-25T20:10:38.768 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047475800000000004 2024-04-25T20:10:38.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.768 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.768 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:38.768 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130751+0000", 2024-04-25T20:10:38.768 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2283000000000002e-05 2024-04-25T20:10:38.768 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.768 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.768 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.768 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.768 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:140 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'98, mlcod=17'87)", 2024-04-25T20:10:38.768 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:38.768 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 80.558174781000005, 2024-04-25T20:10:38.768 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048310990000000002, 2024-04-25T20:10:38.768 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:38.769 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:38.769 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:38.769 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.769 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:38.769 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:38.769 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.769 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.769 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.769 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:38.769 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:38.769 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.769 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.769 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.769 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:38.769 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:38.769 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:38.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.770 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.770 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:38.770 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126427+0000", 2024-04-25T20:10:38.770 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000105572 2024-04-25T20:10:38.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.770 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.770 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:38.770 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126435+0000", 2024-04-25T20:10:38.770 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1920000000000005e-06 2024-04-25T20:10:38.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.771 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.771 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:38.771 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126439+0000", 2024-04-25T20:10:38.771 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6950000000000002e-06 2024-04-25T20:10:38.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.771 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.771 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:38.771 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126471+0000", 2024-04-25T20:10:38.771 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1699000000000003e-05 2024-04-25T20:10:38.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.771 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.771 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:38.771 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126484+0000", 2024-04-25T20:10:38.771 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.343e-05 2024-04-25T20:10:38.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.772 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.772 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:38.772 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131131+0000", 2024-04-25T20:10:38.772 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046471480000000003 2024-04-25T20:10:38.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.772 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.772 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:38.772 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131153+0000", 2024-04-25T20:10:38.772 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1753e-05 2024-04-25T20:10:38.772 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.772 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.772 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.773 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.773 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:141 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'99, mlcod=17'87)", 2024-04-25T20:10:38.773 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:38.773 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 80.557519837000001, 2024-04-25T20:10:38.773 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.004496847, 2024-04-25T20:10:38.773 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:38.773 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:38.773 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:38.773 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.773 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:38.773 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:38.773 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.773 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.774 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:38.774 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:38.774 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:38.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.774 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.774 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:38.774 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:38.774 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:38.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.774 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.774 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:38.774 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126980+0000", 2024-04-25T20:10:38.774 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.619e-06 2024-04-25T20:10:38.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.775 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:38.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126987+0000", 2024-04-25T20:10:38.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7220000000000002e-06 2024-04-25T20:10:38.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.775 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:38.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126992+0000", 2024-04-25T20:10:38.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8679999999999998e-06 2024-04-25T20:10:38.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.775 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.775 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:38.775 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130772+0000", 2024-04-25T20:10:38.775 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037803120000000001 2024-04-25T20:10:38.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:38.776 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130786+0000", 2024-04-25T20:10:38.776 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3592e-05 2024-04-25T20:10:38.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.776 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:38.776 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131451+0000", 2024-04-25T20:10:38.776 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000665206 2024-04-25T20:10:38.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:38.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:38.777 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:38.777 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131474+0000", 2024-04-25T20:10:38.777 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2957999999999999e-05 2024-04-25T20:10:38.777 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.777 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.777 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.777 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:38.777 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:38.777 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:38.779 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T20:10:38.779 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:38.778+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:38.868 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-25T20:10:38.879 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:38.879+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:38.895 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:38.895 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:38.895 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:38.895 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:38.907 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T20:10:38.915 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:10:38.915 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-04-25T20:10:38.937 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2024-04-25T20:10:37.614737+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":173904,"kb_used_data":3588,"kb_used_omap":9,"kb_used_meta":170230,"kb_avail":566057136,"statfs":{"total":579820584960,"available":579642507264,"internally_reserved":0,"allocated":3674112,"data_stored":2630385,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9555,"internal_metadata":174316205},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.862305"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":52,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T20:10:33.445919+0000","last_change":"2024-04-25T20:09:41.160555+0000","last_active":"2024-04-25T20:10:33.445919+0000","last_peered":"2024-04-25T20:10:33.445919+0000","last_clean":"2024-04-25T20:10:33.445919+0000","last_became_active":"2024-04-25T20:09:41.160309+0000","last_became_peered":"2024-04-25T20:09:41.160309+0000","last_unstale":"2024-04-25T20:10:33.445919+0000","last_undegraded":"2024-04-25T20:10:33.445919+0000","last_fullsized":"2024-04-25T20:10:33.445919+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:21:47.019303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":44,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T20:10:33.445927+0000","last_change":"2024-04-25T20:09:16.141209+0000","last_active":"2024-04-25T20:10:33.445927+0000","last_peered":"2024-04-25T20:10:33.445927+0000","last_clean":"2024-04-25T20:10:33.445927+0000","last_became_active":"2024-04-25T20:09:16.140616+0000","last_became_peered":"2024-04-25T20:09:16.140616+0000","last_unstale":"2024-04-25T20:10:33.445927+0000","last_undegraded":"2024-04-25T20:10:33.445927+0000","last_fullsized":"2024-04-25T20:10:33.445927+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:31.562175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T20:10:32.749852+0000","last_change":"2024-04-25T20:10:03.784634+0000","last_active":"2024-04-25T20:10:32.749852+0000","last_peered":"2024-04-25T20:10:32.749852+0000","last_clean":"2024-04-25T20:10:32.749852+0000","last_became_active":"2024-04-25T20:10:03.784247+0000","last_became_peered":"2024-04-25T20:10:03.784247+0000","last_unstale":"2024-04-25T20:10:32.749852+0000","last_undegraded":"2024-04-25T20:10:32.749852+0000","last_fullsized":"2024-04-25T20:10:32.749852+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:35.219740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T20:10:30.730040+0000","last_change":"2024-04-25T20:09:41.938573+0000","last_active":"2024-04-25T20:10:30.730040+0000","last_peered":"2024-04-25T20:10:30.730040+0000","last_clean":"2024-04-25T20:10:30.730040+0000","last_became_active":"2024-04-25T20:09:41.938314+0000","last_became_peered":"2024-04-25T20:09:41.938314+0000","last_unstale":"2024-04-25T20:10:30.730040+0000","last_undegraded":"2024-04-25T20:10:30.730040+0000","last_fullsized":"2024-04-25T20:10:30.730040+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:01:12.669404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":192,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T20:10:30.730013+0000","last_change":"2024-04-25T20:09:17.121148+0000","last_active":"2024-04-25T20:10:30.730013+0000","last_peered":"2024-04-25T20:10:30.730013+0000","last_clean":"2024-04-25T20:10:30.730013+0000","last_became_active":"2024-04-25T20:09:17.120312+0000","last_became_peered":"2024-04-25T20:09:17.120312+0000","last_unstale":"2024-04-25T20:10:30.730013+0000","last_undegraded":"2024-04-25T20:10:30.730013+0000","last_fullsized":"2024-04-25T20:10:30.730013+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_clean_scrub_stamp":"2024-04-25T20:09:15.745353+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:48.383841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":45,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T20:10:32.762922+0000","last_change":"2024-04-25T20:09:15.772839+0000","last_active":"2024-04-25T20:10:32.762922+0000","last_peered":"2024-04-25T20:10:32.762922+0000","last_clean":"2024-04-25T20:10:32.762922+0000","last_became_active":"2024-04-25T20:09:15.772551+0000","last_became_peered":"2024-04-25T20:09:15.772551+0000","last_unstale":"2024-04-25T20:10:32.762922+0000","last_undegraded":"2024-04-25T20:10:32.762922+0000","last_fullsized":"2024-04-25T20:10:32.762922+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:55.101025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":3,"reported_epoch":52,"state":"peering","last_fresh":"2024-04-25T20:10:33.809485+0000","last_change":"2024-04-25T20:10:33.809485+0000","last_active":"2024-04-25T20:09:14.743556+0000","last_peered":"2024-04-25T20:09:14.743556+0000","last_clean":"2024-04-25T20:09:14.743556+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T20:10:33.809485+0000","last_undegraded":"2024-04-25T20:10:33.809485+0000","last_fullsized":"2024-04-25T20:10:33.809485+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:41.384990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T20:10:31.393481+0000","last_change":"2024-04-25T20:09:35.912960+0000","last_active":"2024-04-25T20:10:31.393481+0000","last_peered":"2024-04-25T20:10:31.393481+0000","last_clean":"2024-04-25T20:10:31.393481+0000","last_became_active":"2024-04-25T20:09:35.912678+0000","last_became_peered":"2024-04-25T20:09:35.912678+0000","last_unstale":"2024-04-25T20:10:31.393481+0000","last_undegraded":"2024-04-25T20:10:31.393481+0000","last_fullsized":"2024-04-25T20:10:31.393481+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:28.437601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":43,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T20:10:30.730017+0000","last_change":"2024-04-25T20:09:16.141064+0000","last_active":"2024-04-25T20:10:30.730017+0000","last_peered":"2024-04-25T20:10:30.730017+0000","last_clean":"2024-04-25T20:10:30.730017+0000","last_became_active":"2024-04-25T20:09:16.140731+0000","last_became_peered":"2024-04-25T20:09:16.140731+0000","last_unstale":"2024-04-25T20:10:30.730017+0000","last_undegraded":"2024-04-25T20:10:30.730017+0000","last_fullsized":"2024-04-25T20:10:30.730017+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:18.458296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":655360,"data_stored":833951,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.495}]},{"osd":1,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.51300000000000001}]},{"osd":2,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.68000000000000005}]},{"osd":3,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.60999999999999999}]},{"osd":4,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.59399999999999997}]},{"osd":6,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.66000000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":598016,"data_stored":243583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.44600000000000001}]},{"osd":1,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.439},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.55900000000000005}]},{"osd":2,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.53300000000000003}]},{"osd":3,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.374}]},{"osd":4,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.48599999999999999}]},{"osd":5,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.307},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.45700000000000002}]},{"osd":7,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.47599999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":598016,"data_stored":243583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.39400000000000002}]},{"osd":1,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.379},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.32500000000000001}]},{"osd":2,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.53900000000000003}]},{"osd":3,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.55000000000000004}]},{"osd":4,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.51300000000000001}]},{"osd":6,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.48299999999999998}]},{"osd":7,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.434},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.499}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":598016,"data_stored":243583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.5},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.38500000000000001}]},{"osd":1,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.47499999999999998}]},{"osd":2,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.36099999999999999}]},{"osd":3,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.40000000000000002}]},{"osd":5,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.315}]},{"osd":6,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.52400000000000002}]},{"osd":7,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.34599999999999997}]}]},{"osd":3,"up_from":0,"seq":55834574868,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":0,"seq":55834574867,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31756,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340084,"statfs":{"total":96636764160,"available":96604246016,"internally_reserved":0,"allocated":655360,"data_stored":833951,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":3.0569999999999999,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":2.9660000000000002,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":0.30099999999999999}]},{"osd":2,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.0289999999999999,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.53100000000000003}]},{"osd":3,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.0859999999999999,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.9950000000000001,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":0.61199999999999999}]},{"osd":4,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.438},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.63300000000000001}]},{"osd":5,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.503},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.373}]},{"osd":6,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.57799999999999996}]},{"osd":7,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.47799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":569344,"data_stored":231734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Apr 25 20:10:15 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"last":0.624}]},{"osd":2,"last update":"Thu Apr 25 20:10:15 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.66600000000000004}]},{"osd":3,"last update":"Thu Apr 25 20:10:15 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.496},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.64100000000000001}]},{"osd":4,"last update":"Thu Apr 25 20:10:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.65500000000000003}]},{"osd":5,"last update":"Thu Apr 25 20:10:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.76400000000000001}]},{"osd":6,"last update":"Thu Apr 25 20:10:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.69899999999999995}]},{"osd":7,"last update":"Thu Apr 25 20:10:15 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.755}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:10:38.939 INFO:tasks.thrashosds.thrasher:Setting 1.7 to [3, 6, 7, 1] 2024-04-25T20:10:38.939 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '1.7', '3', '6', '7', '1'] 2024-04-25T20:10:38.940 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 1.7 3 6 7 1 2024-04-25T20:10:38.981 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:38.980+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:39.081 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:39.081+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:39.086 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:39.086 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:39.087 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:39.087 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:39.087 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:39.100 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T20:10:39.183 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:39.183+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:39.283 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:39.283+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:39.319 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:39.319 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:39.320 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:39.320 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:10:39.320 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:39.332 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T20:10:39.416 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:39.416+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:39.479 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:39.480 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:39.480 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:39.480 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:39.484 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:39.484+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:39.489 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T20:10:39.585 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:39.585+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:39.666 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:39.666 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:39.666 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:39.666 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:39.666 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:39.678 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T20:10:39.686 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:39.686+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:39.785 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:39.785 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:39.785 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:39.785 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:10:39.785 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:39.787 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:39.787+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:39.794 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T20:10:39.888 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:39.888+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:39.908 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:39.908 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:39.908 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:10:39.908 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:39.915 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is set 2024-04-25T20:10:39.915 INFO:teuthology.orchestra.run.smithi039.stderr:set 1.7 pg_upmap_items mapping to [3->6,7->1] 2024-04-25T20:10:39.918 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T20:10:39.989 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:39.988+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:40.020 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:40.020 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:40.020 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:40.020 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:40.020 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:40.029 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T20:10:40.113 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:40.112+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:40.171 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:40.171 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:40.171 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:40.171 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:10:40.171 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:40.180 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T20:10:40.221 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:40.220+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:40.292 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:40.291+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:40.321 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:40.322 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:40.322 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:10:40.322 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:40.330 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T20:10:40.414 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:40.413+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:40.417 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T20:10:40.471 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:40.471 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:40.471 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:40.471 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:40.471 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:40.480 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T20:10:40.494 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:40.494+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:40.562 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:10:40.562 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:10:40.590 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:10:40.590 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:10:40.595 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:40.595+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:40.606 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:10:40.607 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:10:40.621 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:40.621 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:40.622 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:40.622 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:10:40.622 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:40.623 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:10:40.623 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:10:40.630 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T20:10:40.640 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:10:40.641 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:10:40.671 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:10:40.672 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:10:40.688 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:10:40.689 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:10:40.696 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:40.696+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:40.720 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:10:40.720 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:10:40.733 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:40.733 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:40.742 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T20:10:40.797 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:40.796+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:40.843 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:40.843 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:40.852 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T20:10:40.936 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:40.935+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:40.996 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:40.996 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:40.998 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:40.997+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:41.005 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T20:10:41.099 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:41.099+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:41.148 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:41.148 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:41.156 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T20:10:41.200 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:41.199+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:41.257 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:41.258 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:41.266 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T20:10:41.301 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:41.300+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:41.401 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:41.401+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:41.409 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:41.409 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:41.418 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T20:10:41.502 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:41.501+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:41.524 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:41.524 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:41.533 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T20:10:41.603 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:41.603+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:41.680 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:41.680 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:41.689 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T20:10:41.733 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:41.733+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:41.798 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:41.798 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:41.805 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:41.805+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:41.807 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T20:10:41.906 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:41.906+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:41.934 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-25T20:10:41.957 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:41.957 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:41.968 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T20:10:42.009 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:42.009+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:42.108 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:42.107+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:42.119 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:42.119 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:42.128 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T20:10:42.212 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:42.212+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:42.276 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:42.276 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:42.285 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T20:10:42.310 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:42.310+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:42.411 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:42.411+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:42.432 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:42.432 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:42.442 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T20:10:42.512 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:42.512+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:42.549 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:42.549 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:42.558 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T20:10:42.613 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:42.613+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:42.666 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:42.666 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:42.675 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T20:10:42.718 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:42.718+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:42.734 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T20:10:42.784 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:42.784 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:42.793 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T20:10:42.815 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:42.814+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:42.860 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:10:42.860 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:10:42.877 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:10:42.878 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:10:42.894 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:10:42.895 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:10:42.903 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:42.903 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:42.913 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T20:10:42.913 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:10:42.914 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:10:42.916 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:42.915+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:42.932 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:10:42.932 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:10:42.937 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is set 2024-04-25T20:10:42.952 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:10:42.952 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:10:42.970 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:10:42.971 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:10:42.988 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:10:42.988 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:10:43.017 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:43.017+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:43.026 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:43.026 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:43.026 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:43.026 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:10:43.026 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:43.036 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T20:10:43.118 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:43.117+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:43.151 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:43.151 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:43.151 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:10:43.151 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:43.161 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T20:10:43.244 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:43.244+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:43.297 INFO:tasks.workunit.client.0.smithi039.stderr:Updating files: 90% (11309/12492) Updating files: 91% (11368/12492) Updating files: 92% (11493/12492) Updating files: 93% (11618/12492) Updating files: 94% (11743/12492) Updating files: 95% (11868/12492) Updating files: 96% (11993/12492) Updating files: 97% (12118/12492) Updating files: 98% (12243/12492) Updating files: 99% (12368/12492) Updating files: 100% (12492/12492) Updating files: 100% (12492/12492), done. 2024-04-25T20:10:43.302 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:43.303 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:43.303 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:43.303 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:43.303 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:43.311 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T20:10:43.318 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:43.318+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:43.419 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:43.419+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:43.453 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:43.453 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:43.453 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:43.453 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:10:43.453 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:43.463 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T20:10:43.520 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:43.519+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:43.565 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:43.565 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:43.565 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:10:43.565 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:43.574 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T20:10:43.620 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:43.620+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:43.714 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:43.715 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:43.715 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:43.715 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:43.715 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:43.721 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:43.720+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:43.723 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T20:10:43.822 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:43.822+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:43.864 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:43.864 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:43.864 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:43.864 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:10:43.865 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:43.874 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T20:10:43.923 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:43.922+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:43.973 INFO:tasks.workunit.client.0.smithi039.stderr:Note: switching to '19b26365dac54ee709df8c5dc2a7b7e89a50aba5'. 2024-04-25T20:10:43.973 INFO:tasks.workunit.client.0.smithi039.stderr: 2024-04-25T20:10:43.974 INFO:tasks.workunit.client.0.smithi039.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T20:10:43.974 INFO:tasks.workunit.client.0.smithi039.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T20:10:43.974 INFO:tasks.workunit.client.0.smithi039.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T20:10:43.974 INFO:tasks.workunit.client.0.smithi039.stderr: 2024-04-25T20:10:43.974 INFO:tasks.workunit.client.0.smithi039.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T20:10:43.974 INFO:tasks.workunit.client.0.smithi039.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T20:10:43.974 INFO:tasks.workunit.client.0.smithi039.stderr: 2024-04-25T20:10:43.974 INFO:tasks.workunit.client.0.smithi039.stderr: git switch -c 2024-04-25T20:10:43.974 INFO:tasks.workunit.client.0.smithi039.stderr: 2024-04-25T20:10:43.974 INFO:tasks.workunit.client.0.smithi039.stderr:Or undo this operation with: 2024-04-25T20:10:43.974 INFO:tasks.workunit.client.0.smithi039.stderr: 2024-04-25T20:10:43.974 INFO:tasks.workunit.client.0.smithi039.stderr: git switch - 2024-04-25T20:10:43.974 INFO:tasks.workunit.client.0.smithi039.stderr: 2024-04-25T20:10:43.974 INFO:tasks.workunit.client.0.smithi039.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T20:10:43.974 INFO:tasks.workunit.client.0.smithi039.stderr: 2024-04-25T20:10:43.975 INFO:tasks.workunit.client.0.smithi039.stderr:HEAD is now at 19b26365dac Merge pull request #56541 from cybozu/wip-65172-reef 2024-04-25T20:10:43.976 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:43.976 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:43.976 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:10:43.976 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:43.980 DEBUG:teuthology.orchestra.run.smithi039:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T20:10:43.985 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T20:10:44.028 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:44.027+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:44.043 INFO:tasks.workunit.client.0.smithi039.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T20:10:44.045 INFO:tasks.workunit.client.0.smithi039.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T20:10:44.045 INFO:tasks.workunit.client.0.smithi039.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T20:10:44.086 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:44.087 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:44.087 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:44.087 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:44.087 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:44.095 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T20:10:44.124 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:44.123+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:44.225 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:44.224+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:44.238 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:44.238 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:44.238 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:44.238 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [] 2024-04-25T20:10:44.238 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:44.247 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T20:10:44.326 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:44.325+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:44.348 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:44.348 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:44.348 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:10:44.348 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:44.357 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T20:10:44.441 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:44.440+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:44.498 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:44.498 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:44.498 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:44.499 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:44.499 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:44.507 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T20:10:44.548 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:44.547+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:44.628 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:44.628+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:44.650 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:44.650 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:44.650 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:44.650 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [ 2024-04-25T20:10:44.650 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.651 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:44.651 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:10:44.651 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 87.866520442999999, 2024-04-25T20:10:44.651 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.34154860100000001, 2024-04-25T20:10:44.651 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:44.651 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:44.651 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:44.651 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:44.651 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:44.651 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 1 2024-04-25T20:10:44.651 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.651 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:44.651 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.651 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:44.651 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:10:44.652 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.652 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.652 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.652 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:44.652 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:10:44.652 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.652 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.652 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.652 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:44.652 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782261+0000", 2024-04-25T20:10:44.652 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999986 2024-04-25T20:10:44.652 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.652 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.652 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:44.652 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782268+0000", 2024-04-25T20:10:44.652 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.8140000000000001e-06 2024-04-25T20:10:44.653 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.653 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.653 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:44.653 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782279+0000", 2024-04-25T20:10:44.653 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0665999999999999e-05 2024-04-25T20:10:44.653 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.653 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.653 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:44.653 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782290+0000", 2024-04-25T20:10:44.653 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.1343e-05 2024-04-25T20:10:44.653 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.653 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.653 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:44.653 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782340+0000", 2024-04-25T20:10:44.653 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.9311999999999998e-05 2024-04-25T20:10:44.653 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.654 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.654 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for peered", 2024-04-25T20:10:44.654 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782348+0000", 2024-04-25T20:10:44.654 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.0690000000000002e-06 2024-04-25T20:10:44.654 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.654 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.654 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:44.654 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120520+0000", 2024-04-25T20:10:44.654 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.33817188100000001 2024-04-25T20:10:44.654 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.654 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.654 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:44.654 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120656+0000", 2024-04-25T20:10:44.655 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000135897 2024-04-25T20:10:44.655 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.655 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.655 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:44.655 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120802+0000", 2024-04-25T20:10:44.655 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00014691199999999999 2024-04-25T20:10:44.655 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.655 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.655 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:44.655 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.122368+0000", 2024-04-25T20:10:44.655 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0015651040000000001 2024-04-25T20:10:44.655 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.655 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.655 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:44.655 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123729+0000", 2024-04-25T20:10:44.655 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0013610499999999999 2024-04-25T20:10:44.656 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.656 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.656 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:44.656 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123769+0000", 2024-04-25T20:10:44.656 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0719000000000001e-05 2024-04-25T20:10:44.656 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.656 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.656 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:44.656 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123811+0000", 2024-04-25T20:10:44.656 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1267999999999999e-05 2024-04-25T20:10:44.656 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.656 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:44.656 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.656 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.656 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.657 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:67 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:44.657 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:10:44.657 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 87.437101556000002, 2024-04-25T20:10:44.657 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.006947651, 2024-04-25T20:10:44.657 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:44.657 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:44.657 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:44.657 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:44.657 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:44.657 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 67 2024-04-25T20:10:44.657 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.657 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:44.657 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.657 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:44.657 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:10:44.658 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.658 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.658 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.658 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:44.658 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:10:44.658 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.658 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.658 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.658 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:44.658 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211680+0000", 2024-04-25T20:10:44.658 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:44.658 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.658 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.658 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:44.658 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211684+0000", 2024-04-25T20:10:44.659 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4249999999999998e-06 2024-04-25T20:10:44.659 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.659 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.659 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:44.659 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211690+0000", 2024-04-25T20:10:44.659 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.7360000000000004e-06 2024-04-25T20:10:44.659 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.659 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.659 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:44.659 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211694+0000", 2024-04-25T20:10:44.659 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4860000000000002e-06 2024-04-25T20:10:44.659 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.659 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.659 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:44.659 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211727+0000", 2024-04-25T20:10:44.659 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3445999999999998e-05 2024-04-25T20:10:44.659 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.660 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.660 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:44.660 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211778+0000", 2024-04-25T20:10:44.660 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1050000000000001e-05 2024-04-25T20:10:44.660 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.660 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.660 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:44.660 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211851+0000", 2024-04-25T20:10:44.660 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2914e-05 2024-04-25T20:10:44.660 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.660 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.660 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:44.660 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213824+0000", 2024-04-25T20:10:44.660 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0019726520000000001 2024-04-25T20:10:44.660 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.660 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.660 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:44.661 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218579+0000", 2024-04-25T20:10:44.661 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0047553159999999999 2024-04-25T20:10:44.661 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.661 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.661 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:44.661 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218607+0000", 2024-04-25T20:10:44.661 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8144999999999999e-05 2024-04-25T20:10:44.661 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.661 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.661 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:44.661 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218629+0000", 2024-04-25T20:10:44.661 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.1044e-05 2024-04-25T20:10:44.661 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.661 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:44.661 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.661 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.661 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.662 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:68 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:44.662 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:44.662 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 87.437083215000001, 2024-04-25T20:10:44.662 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012255596000000001, 2024-04-25T20:10:44.662 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:44.662 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:44.662 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:44.662 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:44.662 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:44.662 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 68 2024-04-25T20:10:44.662 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.662 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:44.662 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.662 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:44.663 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:44.663 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.663 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.663 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.663 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:44.663 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:44.663 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.663 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.663 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.663 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:44.663 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:44.663 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:44.663 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.663 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.663 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:44.663 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211701+0000", 2024-04-25T20:10:44.663 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2029999999999999e-06 2024-04-25T20:10:44.664 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.664 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.664 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:44.664 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211705+0000", 2024-04-25T20:10:44.664 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.2320000000000003e-06 2024-04-25T20:10:44.664 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.664 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.664 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:44.664 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211708+0000", 2024-04-25T20:10:44.664 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.6290000000000002e-06 2024-04-25T20:10:44.664 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.664 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.664 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:44.664 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212136+0000", 2024-04-25T20:10:44.664 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00042764099999999998 2024-04-25T20:10:44.664 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.665 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.665 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:44.665 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212212+0000", 2024-04-25T20:10:44.665 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.6315000000000004e-05 2024-04-25T20:10:44.665 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.665 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.665 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:44.665 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212306+0000", 2024-04-25T20:10:44.665 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.3556999999999999e-05 2024-04-25T20:10:44.665 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.665 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.665 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:44.665 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213829+0000", 2024-04-25T20:10:44.665 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001523256 2024-04-25T20:10:44.665 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.665 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.666 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:44.666 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223904+0000", 2024-04-25T20:10:44.666 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010075387 2024-04-25T20:10:44.666 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.666 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.666 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:44.666 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223934+0000", 2024-04-25T20:10:44.666 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9988000000000002e-05 2024-04-25T20:10:44.666 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.666 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.666 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:44.666 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223955+0000", 2024-04-25T20:10:44.666 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0630999999999999e-05 2024-04-25T20:10:44.666 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.666 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:44.666 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.667 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.667 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.667 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:69 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:44.667 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:44.667 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 87.437061479999997, 2024-04-25T20:10:44.667 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012316863000000001, 2024-04-25T20:10:44.667 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:44.667 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:44.667 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:44.667 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:44.667 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:44.667 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 69 2024-04-25T20:10:44.667 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.667 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:44.668 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.668 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:44.668 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:44.668 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.668 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.668 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.668 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:44.668 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:44.668 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.668 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.668 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.668 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:44.668 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:44.668 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:44.668 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.669 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.669 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:44.669 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211762+0000", 2024-04-25T20:10:44.669 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1703999999999999e-05 2024-04-25T20:10:44.669 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.669 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.669 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:44.669 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211768+0000", 2024-04-25T20:10:44.669 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.3430000000000003e-06 2024-04-25T20:10:44.669 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.669 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.669 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:44.669 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211771+0000", 2024-04-25T20:10:44.669 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.18e-06 2024-04-25T20:10:44.669 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.670 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.670 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:44.670 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212652+0000", 2024-04-25T20:10:44.670 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00088095800000000004 2024-04-25T20:10:44.670 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.670 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.670 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:44.670 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212703+0000", 2024-04-25T20:10:44.670 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.0655999999999997e-05 2024-04-25T20:10:44.670 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.670 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.670 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:44.670 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212781+0000", 2024-04-25T20:10:44.670 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.8852999999999999e-05 2024-04-25T20:10:44.671 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.671 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.671 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:44.671 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.214131+0000", 2024-04-25T20:10:44.671 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0013496039999999999 2024-04-25T20:10:44.671 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.671 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.671 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:44.671 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223986+0000", 2024-04-25T20:10:44.671 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.009854804 2024-04-25T20:10:44.671 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.671 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.671 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:44.671 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.224017+0000", 2024-04-25T20:10:44.671 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.1068999999999997e-05 2024-04-25T20:10:44.672 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.672 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.672 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:44.672 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.224038+0000", 2024-04-25T20:10:44.672 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0965e-05 2024-04-25T20:10:44.672 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.672 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:44.672 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.672 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.672 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.672 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:70 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:44.672 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:44.672 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 87.437006398999998, 2024-04-25T20:10:44.672 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.015776348999999999, 2024-04-25T20:10:44.672 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:44.673 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:44.673 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:44.673 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:44.673 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:44.673 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 70 2024-04-25T20:10:44.673 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.673 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:44.673 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.673 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:44.673 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:44.673 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.673 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.673 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.673 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:44.673 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:44.673 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.674 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.674 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.674 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:44.674 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:44.674 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:44.674 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.674 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.674 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:44.674 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211778+0000", 2024-04-25T20:10:44.674 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5330000000000001e-06 2024-04-25T20:10:44.674 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.674 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.674 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:44.674 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211782+0000", 2024-04-25T20:10:44.675 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4079999999999998e-06 2024-04-25T20:10:44.675 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.675 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.675 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:44.675 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211786+0000", 2024-04-25T20:10:44.675 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.6190000000000002e-06 2024-04-25T20:10:44.675 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.675 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.675 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:44.675 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213374+0000", 2024-04-25T20:10:44.675 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0015881859999999999 2024-04-25T20:10:44.675 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.675 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.675 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:44.675 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213478+0000", 2024-04-25T20:10:44.675 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000103464 2024-04-25T20:10:44.675 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.676 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.676 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:44.676 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213601+0000", 2024-04-25T20:10:44.676 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00012324300000000001 2024-04-25T20:10:44.676 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.676 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.676 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:44.676 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.214788+0000", 2024-04-25T20:10:44.676 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011868569999999999 2024-04-25T20:10:44.676 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.676 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.676 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:44.676 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227504+0000", 2024-04-25T20:10:44.676 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012715749 2024-04-25T20:10:44.676 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.677 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.677 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:44.677 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227532+0000", 2024-04-25T20:10:44.677 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.7937e-05 2024-04-25T20:10:44.677 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.677 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.677 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:44.677 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227552+0000", 2024-04-25T20:10:44.677 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0772999999999999e-05 2024-04-25T20:10:44.677 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.677 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:44.677 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.677 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.677 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.677 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:131 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:44.677 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:10:44.678 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 86.527996553999998, 2024-04-25T20:10:44.678 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0086158810000000006, 2024-04-25T20:10:44.678 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:44.678 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:44.678 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:44.678 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:44.678 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:44.678 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 131 2024-04-25T20:10:44.678 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.678 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:44.678 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.678 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:44.678 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:10:44.678 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.678 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.679 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.679 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:44.679 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:10:44.679 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.679 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.679 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.679 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:44.679 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120785+0000", 2024-04-25T20:10:44.679 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:44.679 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.679 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.679 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:44.679 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120880+0000", 2024-04-25T20:10:44.679 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.4925000000000003e-05 2024-04-25T20:10:44.679 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.679 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.680 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:44.680 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120891+0000", 2024-04-25T20:10:44.680 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0915e-05 2024-04-25T20:10:44.680 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.680 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.680 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:44.680 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120895+0000", 2024-04-25T20:10:44.680 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.8070000000000001e-06 2024-04-25T20:10:44.680 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.680 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.680 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:44.680 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121640+0000", 2024-04-25T20:10:44.680 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00074468500000000003 2024-04-25T20:10:44.680 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.680 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.680 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:44.680 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121684+0000", 2024-04-25T20:10:44.681 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4452000000000001e-05 2024-04-25T20:10:44.681 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.681 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.681 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:44.681 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121753+0000", 2024-04-25T20:10:44.681 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8508000000000002e-05 2024-04-25T20:10:44.681 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.681 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.681 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:44.681 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123618+0000", 2024-04-25T20:10:44.681 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0018655099999999999 2024-04-25T20:10:44.681 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.681 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.681 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:44.681 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129248+0000", 2024-04-25T20:10:44.681 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0056295540000000002 2024-04-25T20:10:44.682 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.682 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.682 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:44.682 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129368+0000", 2024-04-25T20:10:44.682 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000120244 2024-04-25T20:10:44.682 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.682 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.682 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:44.682 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129402+0000", 2024-04-25T20:10:44.682 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3812000000000001e-05 2024-04-25T20:10:44.682 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.682 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:44.682 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.682 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.682 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.682 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:132 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:44.683 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:44.683 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 86.527882344000005, 2024-04-25T20:10:44.683 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0085898169999999996, 2024-04-25T20:10:44.683 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:44.683 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:44.683 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:44.683 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:44.683 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:44.683 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 132 2024-04-25T20:10:44.683 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.683 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:44.683 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.683 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:44.683 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:44.683 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.684 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.684 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.684 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:44.684 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:44.684 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.684 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.684 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.684 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:44.684 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:44.684 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:44.684 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.684 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.684 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:44.684 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120902+0000", 2024-04-25T20:10:44.684 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.6129999999999998e-06 2024-04-25T20:10:44.685 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.685 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.685 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:44.685 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120906+0000", 2024-04-25T20:10:44.685 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6569999999999999e-06 2024-04-25T20:10:44.685 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.685 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.685 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:44.685 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120909+0000", 2024-04-25T20:10:44.685 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5550000000000001e-06 2024-04-25T20:10:44.685 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.685 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.685 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:44.685 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122098+0000", 2024-04-25T20:10:44.685 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011898049999999999 2024-04-25T20:10:44.685 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.686 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.686 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:44.686 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122172+0000", 2024-04-25T20:10:44.686 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3917000000000004e-05 2024-04-25T20:10:44.686 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.686 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.686 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:44.686 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122288+0000", 2024-04-25T20:10:44.686 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00011566 2024-04-25T20:10:44.689 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.689 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.689 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:44.689 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124519+0000", 2024-04-25T20:10:44.690 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0022308359999999999 2024-04-25T20:10:44.690 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.690 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.690 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:44.690 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129414+0000", 2024-04-25T20:10:44.690 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048953720000000003 2024-04-25T20:10:44.690 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.690 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.690 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:44.690 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129458+0000", 2024-04-25T20:10:44.690 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.3426000000000001e-05 2024-04-25T20:10:44.690 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.690 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.691 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:44.691 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129490+0000", 2024-04-25T20:10:44.691 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2329999999999997e-05 2024-04-25T20:10:44.691 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.691 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:44.691 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.691 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.691 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.691 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:133 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:44.691 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:44.691 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 86.527828475000007, 2024-04-25T20:10:44.691 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0086066270000000004, 2024-04-25T20:10:44.691 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:44.691 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:44.691 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:44.692 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:44.692 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:44.692 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 133 2024-04-25T20:10:44.692 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.692 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:44.692 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.692 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:44.692 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:44.692 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.692 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.692 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.692 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:44.692 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:44.692 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.692 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.692 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.692 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:44.693 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:44.693 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:44.693 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.693 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.693 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:44.693 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120956+0000", 2024-04-25T20:10:44.693 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5289999999999998e-06 2024-04-25T20:10:44.693 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.693 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.693 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:44.693 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120962+0000", 2024-04-25T20:10:44.693 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.9490000000000001e-06 2024-04-25T20:10:44.693 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.693 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.693 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:44.693 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120965+0000", 2024-04-25T20:10:44.693 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.1860000000000001e-06 2024-04-25T20:10:44.694 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.694 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.694 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:44.694 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122666+0000", 2024-04-25T20:10:44.694 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0017004500000000001 2024-04-25T20:10:44.694 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.694 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.694 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:44.694 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122741+0000", 2024-04-25T20:10:44.694 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.5297999999999996e-05 2024-04-25T20:10:44.694 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.694 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.694 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:44.694 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122858+0000", 2024-04-25T20:10:44.694 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00011723899999999999 2024-04-25T20:10:44.695 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.695 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.695 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:44.695 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124625+0000", 2024-04-25T20:10:44.695 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0017670100000000001 2024-04-25T20:10:44.695 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.695 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.695 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:44.695 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129497+0000", 2024-04-25T20:10:44.695 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048713580000000001 2024-04-25T20:10:44.695 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.695 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.695 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:44.695 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129532+0000", 2024-04-25T20:10:44.695 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5704999999999998e-05 2024-04-25T20:10:44.696 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.696 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.696 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:44.696 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129561+0000", 2024-04-25T20:10:44.696 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8283e-05 2024-04-25T20:10:44.696 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.696 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:44.696 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.696 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.696 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.696 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:134 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:44.696 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:44.696 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 86.527726276999999, 2024-04-25T20:10:44.696 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0085771179999999999, 2024-04-25T20:10:44.696 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:44.697 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:44.697 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:44.697 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:44.697 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:44.697 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 134 2024-04-25T20:10:44.697 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.697 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:44.697 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.697 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:44.697 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:44.697 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.697 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.697 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.697 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:44.697 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:44.697 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.698 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.698 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.698 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:44.698 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:44.698 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:44.698 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.698 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.698 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:44.698 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121157+0000", 2024-04-25T20:10:44.698 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00010104599999999999 2024-04-25T20:10:44.698 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.698 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.698 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:44.698 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121189+0000", 2024-04-25T20:10:44.698 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2589000000000003e-05 2024-04-25T20:10:44.698 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.699 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.699 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:44.699 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121195+0000", 2024-04-25T20:10:44.699 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.083e-06 2024-04-25T20:10:44.699 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.699 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.699 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:44.699 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123136+0000", 2024-04-25T20:10:44.699 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0019409780000000001 2024-04-25T20:10:44.699 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.699 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.699 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:44.699 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123181+0000", 2024-04-25T20:10:44.699 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4267999999999997e-05 2024-04-25T20:10:44.699 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.699 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.700 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:44.700 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123247+0000", 2024-04-25T20:10:44.700 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.6086000000000004e-05 2024-04-25T20:10:44.700 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.700 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.700 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:44.700 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125639+0000", 2024-04-25T20:10:44.700 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.002392 2024-04-25T20:10:44.700 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.700 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.700 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:44.700 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129567+0000", 2024-04-25T20:10:44.700 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0039282170000000003 2024-04-25T20:10:44.700 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.700 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.700 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:44.701 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129605+0000", 2024-04-25T20:10:44.701 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.7660999999999997e-05 2024-04-25T20:10:44.701 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.701 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.701 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:44.701 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129633+0000", 2024-04-25T20:10:44.701 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8649999999999998e-05 2024-04-25T20:10:44.701 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.701 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:44.701 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.701 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.701 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.701 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:135 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:44.701 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:10:44.701 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 86.527567270999995, 2024-04-25T20:10:44.701 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010898453000000001, 2024-04-25T20:10:44.702 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:44.702 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:44.702 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:44.702 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:44.702 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:44.702 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 135 2024-04-25T20:10:44.702 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.702 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:44.702 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.702 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:44.702 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:10:44.702 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.703 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.703 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.703 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:44.703 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:10:44.703 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.703 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.703 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.703 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:44.703 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121214+0000", 2024-04-25T20:10:44.703 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:44.703 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.703 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.703 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:44.704 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121218+0000", 2024-04-25T20:10:44.704 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6619999999999998e-06 2024-04-25T20:10:44.704 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.704 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.704 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:44.704 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121223+0000", 2024-04-25T20:10:44.704 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1070000000000004e-06 2024-04-25T20:10:44.704 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.704 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.704 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:44.704 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121227+0000", 2024-04-25T20:10:44.704 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2820000000000001e-06 2024-04-25T20:10:44.704 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.704 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.704 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:44.705 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123649+0000", 2024-04-25T20:10:44.705 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0024221970000000001 2024-04-25T20:10:44.705 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.705 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.705 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:44.705 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123723+0000", 2024-04-25T20:10:44.705 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.4386000000000002e-05 2024-04-25T20:10:44.705 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.705 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.705 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:44.705 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123844+0000", 2024-04-25T20:10:44.705 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000121249 2024-04-25T20:10:44.705 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.705 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.706 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:44.706 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125670+0000", 2024-04-25T20:10:44.706 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001825901 2024-04-25T20:10:44.706 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.706 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.706 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:44.706 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132043+0000", 2024-04-25T20:10:44.706 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0063724539999999996 2024-04-25T20:10:44.706 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.706 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.706 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:44.706 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132084+0000", 2024-04-25T20:10:44.706 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0963000000000001e-05 2024-04-25T20:10:44.706 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.707 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.707 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:44.707 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132114+0000", 2024-04-25T20:10:44.707 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9927e-05 2024-04-25T20:10:44.707 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.707 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:44.707 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.707 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.707 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.707 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:136 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:44.707 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:10:44.707 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 86.527551744999997, 2024-04-25T20:10:44.707 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010952513000000001, 2024-04-25T20:10:44.707 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:44.707 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:44.707 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:44.708 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:44.708 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:44.708 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 136 2024-04-25T20:10:44.708 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.708 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:44.708 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.708 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:44.708 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:10:44.708 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.708 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.708 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.708 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:44.708 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:10:44.708 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.708 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.708 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.708 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:44.709 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121230+0000", 2024-04-25T20:10:44.709 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:44.709 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.709 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.709 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:44.709 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121236+0000", 2024-04-25T20:10:44.709 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.7320000000000003e-06 2024-04-25T20:10:44.709 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.709 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.709 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:44.709 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121242+0000", 2024-04-25T20:10:44.709 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.7810000000000002e-06 2024-04-25T20:10:44.709 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.709 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.710 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:44.710 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121246+0000", 2024-04-25T20:10:44.710 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.639e-06 2024-04-25T20:10:44.710 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.710 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.710 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:44.710 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124150+0000", 2024-04-25T20:10:44.710 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.002903816 2024-04-25T20:10:44.710 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.710 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.710 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:44.710 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124195+0000", 2024-04-25T20:10:44.710 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.5042e-05 2024-04-25T20:10:44.710 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.711 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.711 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:44.711 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124266+0000", 2024-04-25T20:10:44.711 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0234999999999994e-05 2024-04-25T20:10:44.711 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.711 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.711 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:44.711 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125711+0000", 2024-04-25T20:10:44.711 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0014458800000000001 2024-04-25T20:10:44.711 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.711 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.711 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:44.711 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132120+0000", 2024-04-25T20:10:44.711 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0064084399999999996 2024-04-25T20:10:44.711 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.712 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.712 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:44.712 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132155+0000", 2024-04-25T20:10:44.712 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4938000000000003e-05 2024-04-25T20:10:44.712 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.712 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.712 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:44.712 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132183+0000", 2024-04-25T20:10:44.712 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8360999999999998e-05 2024-04-25T20:10:44.712 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.712 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:44.712 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.712 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.712 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.713 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:44.713 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:44.713 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 86.527502013000003, 2024-04-25T20:10:44.713 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.01097684, 2024-04-25T20:10:44.713 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:44.713 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:44.713 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:44.713 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:44.713 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:44.713 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 137 2024-04-25T20:10:44.713 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.713 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:44.714 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.714 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:44.714 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:44.714 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.714 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.714 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.714 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:44.714 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:44.714 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.714 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.714 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.714 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:44.714 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:44.715 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:44.715 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.715 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.715 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:44.715 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121328+0000", 2024-04-25T20:10:44.715 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.7908999999999998e-05 2024-04-25T20:10:44.715 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.715 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.715 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:44.715 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121337+0000", 2024-04-25T20:10:44.715 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.9369999999999993e-06 2024-04-25T20:10:44.715 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.715 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.715 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:44.715 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121343+0000", 2024-04-25T20:10:44.715 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.8089999999999998e-06 2024-04-25T20:10:44.716 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.716 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.716 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:44.716 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124656+0000", 2024-04-25T20:10:44.716 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0033133839999999999 2024-04-25T20:10:44.716 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.716 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.716 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:44.716 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124730+0000", 2024-04-25T20:10:44.716 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3565000000000005e-05 2024-04-25T20:10:44.716 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.716 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.716 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:44.717 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124846+0000", 2024-04-25T20:10:44.717 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000116416 2024-04-25T20:10:44.717 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.717 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.717 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:44.717 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126756+0000", 2024-04-25T20:10:44.717 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001909991 2024-04-25T20:10:44.717 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.717 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.717 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:44.717 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132190+0000", 2024-04-25T20:10:44.717 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0054343880000000001 2024-04-25T20:10:44.717 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.717 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.717 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:44.718 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132227+0000", 2024-04-25T20:10:44.718 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6875000000000002e-05 2024-04-25T20:10:44.718 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.718 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.718 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:44.718 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132257+0000", 2024-04-25T20:10:44.718 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.0046e-05 2024-04-25T20:10:44.718 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.718 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:44.718 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.718 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.718 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.718 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:44.718 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:44.719 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 86.527432055000006, 2024-04-25T20:10:44.719 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010978653999999999, 2024-04-25T20:10:44.719 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:44.719 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:44.719 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:44.719 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:44.719 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:44.719 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 138 2024-04-25T20:10:44.719 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.719 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:44.719 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.719 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:44.719 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:44.719 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.720 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.720 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.720 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:44.720 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:44.720 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.720 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.720 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.720 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:44.720 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:44.720 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:44.720 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.720 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.720 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:44.720 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121355+0000", 2024-04-25T20:10:44.720 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.9869999999999999e-06 2024-04-25T20:10:44.720 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.720 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.721 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:44.721 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121361+0000", 2024-04-25T20:10:44.721 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.4620000000000001e-06 2024-04-25T20:10:44.721 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.721 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.721 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:44.721 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121373+0000", 2024-04-25T20:10:44.721 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.2035000000000001e-05 2024-04-25T20:10:44.721 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.721 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.721 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:44.721 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125267+0000", 2024-04-25T20:10:44.721 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0038939230000000001 2024-04-25T20:10:44.721 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.721 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.721 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:44.721 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125309+0000", 2024-04-25T20:10:44.722 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1690999999999998e-05 2024-04-25T20:10:44.722 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.722 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.722 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:44.722 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125385+0000", 2024-04-25T20:10:44.722 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.6162999999999996e-05 2024-04-25T20:10:44.722 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.722 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.722 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:44.722 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126798+0000", 2024-04-25T20:10:44.722 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0014124680000000001 2024-04-25T20:10:44.722 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.722 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.722 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:44.722 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132264+0000", 2024-04-25T20:10:44.723 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0054661689999999999 2024-04-25T20:10:44.723 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.723 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.723 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:44.723 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132299+0000", 2024-04-25T20:10:44.723 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5272999999999999e-05 2024-04-25T20:10:44.723 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.723 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.723 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:44.723 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132329+0000", 2024-04-25T20:10:44.723 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.0080999999999999e-05 2024-04-25T20:10:44.723 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.723 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:44.723 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.723 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.723 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.724 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:44.724 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:10:44.724 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 86.527401479999995, 2024-04-25T20:10:44.724 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011015853000000001, 2024-04-25T20:10:44.724 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:44.724 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:44.724 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:44.724 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:44.724 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:44.724 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 139 2024-04-25T20:10:44.724 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.724 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:44.724 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.724 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:44.724 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:10:44.724 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.725 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.725 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.725 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:44.725 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:10:44.725 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.725 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.725 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.725 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:44.725 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121380+0000", 2024-04-25T20:10:44.725 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:44.725 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.725 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.725 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:44.725 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121385+0000", 2024-04-25T20:10:44.725 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.092e-06 2024-04-25T20:10:44.725 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.725 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.726 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:44.726 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121390+0000", 2024-04-25T20:10:44.726 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.5990000000000001e-06 2024-04-25T20:10:44.726 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.726 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.726 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:44.726 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121397+0000", 2024-04-25T20:10:44.726 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.6279999999999999e-06 2024-04-25T20:10:44.726 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.726 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.726 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:44.726 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125737+0000", 2024-04-25T20:10:44.726 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0043406010000000004 2024-04-25T20:10:44.727 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.727 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.727 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:44.727 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125781+0000", 2024-04-25T20:10:44.727 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.3958999999999997e-05 2024-04-25T20:10:44.727 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.727 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.727 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:44.727 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125849+0000", 2024-04-25T20:10:44.727 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8071999999999995e-05 2024-04-25T20:10:44.727 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.727 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.727 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:44.727 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.130908+0000", 2024-04-25T20:10:44.727 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0050586509999999999 2024-04-25T20:10:44.727 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.728 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.728 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:44.728 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132334+0000", 2024-04-25T20:10:44.728 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00142621 2024-04-25T20:10:44.728 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.728 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.728 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:44.728 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132369+0000", 2024-04-25T20:10:44.728 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4452000000000002e-05 2024-04-25T20:10:44.728 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.728 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.728 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:44.728 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132397+0000", 2024-04-25T20:10:44.728 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8065e-05 2024-04-25T20:10:44.728 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.728 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:44.728 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.729 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.729 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.729 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:44.729 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:44.729 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 86.527380436000001, 2024-04-25T20:10:44.729 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011069443, 2024-04-25T20:10:44.729 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:44.729 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:44.729 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:44.729 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:44.729 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:44.729 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 140 2024-04-25T20:10:44.729 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.729 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:44.729 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.730 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:44.730 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:44.730 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.730 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.730 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.730 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:44.730 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:44.730 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.730 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.730 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.730 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:44.730 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:44.730 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:44.730 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.730 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.730 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:44.731 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121456+0000", 2024-04-25T20:10:44.731 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.4469000000000003e-05 2024-04-25T20:10:44.731 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.731 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.731 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:44.731 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121462+0000", 2024-04-25T20:10:44.731 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.1890000000000004e-06 2024-04-25T20:10:44.731 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.731 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.731 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:44.731 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121466+0000", 2024-04-25T20:10:44.731 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3909999999999998e-06 2024-04-25T20:10:44.731 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.731 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.731 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:44.732 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126071+0000", 2024-04-25T20:10:44.732 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0046048979999999996 2024-04-25T20:10:44.732 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.732 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.732 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:44.732 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126141+0000", 2024-04-25T20:10:44.732 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0621e-05 2024-04-25T20:10:44.732 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.732 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.732 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:44.732 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126260+0000", 2024-04-25T20:10:44.732 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000118621 2024-04-25T20:10:44.732 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.733 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.733 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:44.733 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.131270+0000", 2024-04-25T20:10:44.733 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0050096949999999998 2024-04-25T20:10:44.733 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.733 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.733 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:44.733 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132403+0000", 2024-04-25T20:10:44.733 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011332779999999999 2024-04-25T20:10:44.733 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.733 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.733 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:44.733 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132443+0000", 2024-04-25T20:10:44.734 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0175000000000001e-05 2024-04-25T20:10:44.734 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.734 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.734 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:44.734 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132471+0000", 2024-04-25T20:10:44.734 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8399e-05 2024-04-25T20:10:44.734 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.734 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:44.734 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.734 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.734 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.734 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:44.734 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:10:44.735 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 86.527311651000005, 2024-04-25T20:10:44.735 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011071002, 2024-04-25T20:10:44.735 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:44.735 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:44.735 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:44.735 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:44.735 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:44.735 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 141 2024-04-25T20:10:44.735 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.735 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:44.735 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.735 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:44.735 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:10:44.735 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.735 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.735 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.736 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:44.736 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:10:44.736 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.736 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.736 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.736 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:44.736 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121470+0000", 2024-04-25T20:10:44.736 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:44.736 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.736 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.736 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:44.736 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121473+0000", 2024-04-25T20:10:44.736 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.01e-06 2024-04-25T20:10:44.736 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.736 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.736 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:44.737 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121478+0000", 2024-04-25T20:10:44.737 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.515e-06 2024-04-25T20:10:44.737 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.737 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.737 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:44.737 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121480+0000", 2024-04-25T20:10:44.737 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5749999999999999e-06 2024-04-25T20:10:44.737 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.737 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.737 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:44.737 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126813+0000", 2024-04-25T20:10:44.737 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0053320399999999997 2024-04-25T20:10:44.737 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.737 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.737 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:44.738 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126854+0000", 2024-04-25T20:10:44.738 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1582999999999998e-05 2024-04-25T20:10:44.738 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.738 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.738 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:44.738 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126923+0000", 2024-04-25T20:10:44.738 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8544e-05 2024-04-25T20:10:44.738 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.738 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.738 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:44.738 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.131573+0000", 2024-04-25T20:10:44.738 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0046504659999999998 2024-04-25T20:10:44.738 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.738 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.739 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:44.739 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132478+0000", 2024-04-25T20:10:44.739 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00090442400000000003 2024-04-25T20:10:44.739 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.739 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.739 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:44.739 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132513+0000", 2024-04-25T20:10:44.739 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5204999999999999e-05 2024-04-25T20:10:44.739 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.739 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.739 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:44.739 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132542+0000", 2024-04-25T20:10:44.739 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.904e-05 2024-04-25T20:10:44.739 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.739 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:44.740 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.740 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.740 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.740 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:156 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:44.740 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:44.740 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 86.029487356000004, 2024-04-25T20:10:44.740 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090072280000000008, 2024-04-25T20:10:44.740 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:44.740 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:44.740 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:44.740 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:44.740 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:44.740 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 156 2024-04-25T20:10:44.741 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.741 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:44.741 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.741 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:44.741 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:44.741 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.741 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.741 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.741 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:44.741 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:44.741 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.741 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.741 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.741 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:44.742 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:44.742 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:44.742 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.742 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.742 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:44.742 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619297+0000", 2024-04-25T20:10:44.742 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2979999999999999e-06 2024-04-25T20:10:44.742 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.742 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.742 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:44.742 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619300+0000", 2024-04-25T20:10:44.742 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.371e-06 2024-04-25T20:10:44.742 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.742 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.743 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:44.743 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619304+0000", 2024-04-25T20:10:44.743 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.304e-06 2024-04-25T20:10:44.743 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.743 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.743 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:44.743 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.622910+0000", 2024-04-25T20:10:44.743 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0036065049999999999 2024-04-25T20:10:44.743 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.743 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.743 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:44.743 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.622990+0000", 2024-04-25T20:10:44.743 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.9722999999999995e-05 2024-04-25T20:10:44.743 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.743 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.744 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:44.744 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623128+0000", 2024-04-25T20:10:44.744 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00013818399999999999 2024-04-25T20:10:44.744 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.744 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.744 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:44.744 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625155+0000", 2024-04-25T20:10:44.744 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0020267979999999998 2024-04-25T20:10:44.744 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.744 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.744 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:44.744 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628225+0000", 2024-04-25T20:10:44.744 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0030705680000000001 2024-04-25T20:10:44.745 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.745 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.745 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:44.745 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628268+0000", 2024-04-25T20:10:44.745 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.2404000000000003e-05 2024-04-25T20:10:44.745 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.745 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.745 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:44.745 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628302+0000", 2024-04-25T20:10:44.745 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4439e-05 2024-04-25T20:10:44.745 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.745 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:44.745 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.745 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.745 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.746 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:157 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:44.746 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:44.746 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 86.029475257000001, 2024-04-25T20:10:44.746 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090534889999999996, 2024-04-25T20:10:44.746 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:44.746 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:44.746 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:44.746 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:44.746 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:44.746 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 157 2024-04-25T20:10:44.746 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.746 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:44.746 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.746 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:44.747 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:44.747 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.747 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.747 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.747 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:44.747 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:44.747 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.747 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.747 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.747 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:44.747 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:44.747 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:44.747 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.748 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.748 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:44.748 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619309+0000", 2024-04-25T20:10:44.748 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.841e-06 2024-04-25T20:10:44.748 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.748 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.748 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:44.748 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619312+0000", 2024-04-25T20:10:44.748 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.084e-06 2024-04-25T20:10:44.748 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.748 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.748 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:44.748 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619314+0000", 2024-04-25T20:10:44.748 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.3860000000000001e-06 2024-04-25T20:10:44.749 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.749 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.749 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:44.749 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623718+0000", 2024-04-25T20:10:44.749 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0044038050000000002 2024-04-25T20:10:44.749 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.749 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.749 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:44.749 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623791+0000", 2024-04-25T20:10:44.749 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3181000000000003e-05 2024-04-25T20:10:44.749 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.749 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.749 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:44.749 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623922+0000", 2024-04-25T20:10:44.749 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00013117000000000001 2024-04-25T20:10:44.749 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.750 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.750 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:44.750 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625187+0000", 2024-04-25T20:10:44.750 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001264961 2024-04-25T20:10:44.750 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.750 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.750 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:44.750 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628308+0000", 2024-04-25T20:10:44.750 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.003120739 2024-04-25T20:10:44.750 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.750 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.750 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:44.750 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628331+0000", 2024-04-25T20:10:44.750 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2802999999999998e-05 2024-04-25T20:10:44.750 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.750 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.751 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:44.751 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628361+0000", 2024-04-25T20:10:44.751 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9760999999999998e-05 2024-04-25T20:10:44.751 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.751 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:44.751 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.751 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.751 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.751 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:158 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:44.751 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:10:44.751 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 86.029464809999993, 2024-04-25T20:10:44.751 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0091076409999999997, 2024-04-25T20:10:44.751 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:44.751 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:44.751 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:44.752 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:44.752 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:44.752 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 158 2024-04-25T20:10:44.752 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.752 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:44.752 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.752 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:44.752 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:10:44.752 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.752 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.752 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.752 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:44.752 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:10:44.752 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.752 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.753 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.753 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:44.753 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619317+0000", 2024-04-25T20:10:44.753 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967296 2024-04-25T20:10:44.753 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.753 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.753 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:44.753 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619364+0000", 2024-04-25T20:10:44.753 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.6572000000000001e-05 2024-04-25T20:10:44.753 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.753 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.753 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:44.753 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619371+0000", 2024-04-25T20:10:44.753 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3860000000000001e-06 2024-04-25T20:10:44.754 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.754 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.754 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:44.754 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619382+0000", 2024-04-25T20:10:44.754 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0173999999999999e-05 2024-04-25T20:10:44.754 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.754 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.754 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:44.754 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624216+0000", 2024-04-25T20:10:44.754 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048340350000000004 2024-04-25T20:10:44.754 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.754 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.754 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:44.755 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624260+0000", 2024-04-25T20:10:44.755 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4558999999999998e-05 2024-04-25T20:10:44.755 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.755 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.755 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:44.755 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624333+0000", 2024-04-25T20:10:44.755 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2744999999999996e-05 2024-04-25T20:10:44.755 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.755 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.755 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:44.755 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625465+0000", 2024-04-25T20:10:44.755 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001132454 2024-04-25T20:10:44.755 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.756 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.756 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:44.756 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628365+0000", 2024-04-25T20:10:44.756 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0028994950000000002 2024-04-25T20:10:44.756 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.756 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.756 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:44.756 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628386+0000", 2024-04-25T20:10:44.756 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.1367000000000001e-05 2024-04-25T20:10:44.756 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.756 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.756 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:44.756 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628425+0000", 2024-04-25T20:10:44.756 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.9087e-05 2024-04-25T20:10:44.756 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.757 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:44.757 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.757 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.757 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.757 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:159 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:44.757 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:44.757 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 86.029396156999994, 2024-04-25T20:10:44.757 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090948999999999995, 2024-04-25T20:10:44.757 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:44.757 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:44.757 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:44.757 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:44.757 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:44.757 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 159 2024-04-25T20:10:44.758 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.758 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:44.758 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.758 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:44.758 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:44.758 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.758 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.758 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.758 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:44.758 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:44.758 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:44.758 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.758 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.758 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:44.759 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:44.759 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:44.759 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.759 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.759 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:44.759 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619389+0000", 2024-04-25T20:10:44.759 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.734e-06 2024-04-25T20:10:44.759 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.759 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.759 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:44.759 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619392+0000", 2024-04-25T20:10:44.759 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6219999999999998e-06 2024-04-25T20:10:44.759 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.759 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.760 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:44.760 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619403+0000", 2024-04-25T20:10:44.760 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0987e-05 2024-04-25T20:10:44.760 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.760 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.760 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:44.760 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624667+0000", 2024-04-25T20:10:44.760 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0052633970000000004 2024-04-25T20:10:44.760 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.760 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.760 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:44.760 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624741+0000", 2024-04-25T20:10:44.760 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.4583000000000001e-05 2024-04-25T20:10:44.760 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.760 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.761 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:44.761 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624857+0000", 2024-04-25T20:10:44.761 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000116118 2024-04-25T20:10:44.761 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.761 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.761 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:44.761 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.626013+0000", 2024-04-25T20:10:44.761 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001155473 2024-04-25T20:10:44.761 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.761 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.761 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:44.761 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628432+0000", 2024-04-25T20:10:44.761 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0024189379999999998 2024-04-25T20:10:44.761 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.762 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.762 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:44.762 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628454+0000", 2024-04-25T20:10:44.762 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2059999999999999e-05 2024-04-25T20:10:44.762 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:44.762 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:44.762 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:44.762 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628481+0000", 2024-04-25T20:10:44.762 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.7300999999999998e-05 2024-04-25T20:10:44.762 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.762 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:44.762 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.762 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:44.762 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:44.762 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:44.763 INFO:tasks.workunit.client.0.smithi039.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T20:10:44.764 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:44.763+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:44.765 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T20:10:44.815 INFO:tasks.workunit.client.0.smithi039.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T20:10:44.858 INFO:tasks.workunit.client.0.smithi039.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T20:10:44.859 INFO:tasks.workunit.client.0.smithi039.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T20:10:44.860 INFO:tasks.workunit.client.0.smithi039.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T20:10:44.866 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:44.866+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:44.880 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:44.880 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:44.880 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:44.881 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:44.890 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T20:10:44.900 INFO:tasks.workunit.client.0.smithi039.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T20:10:44.904 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-04-25T20:10:44.904 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T20:10:44.931 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7] out_osds: [3, 2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-25T20:10:44.931 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T20:10:44.932 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-04-25T20:10:44.932 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-04-25T20:10:44.957 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-25T20:10:44.962 INFO:tasks.workunit:Running workunits matching rbd/journal.sh on client.0... 2024-04-25T20:10:44.963 INFO:tasks.workunit:Running workunit rbd/journal.sh... 2024-04-25T20:10:44.964 DEBUG:teuthology.orchestra.run.smithi039:workunit test rbd/journal.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=19b26365dac54ee709df8c5dc2a7b7e89a50aba5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/journal.sh 2024-04-25T20:10:45.000 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:45.000 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:45.000 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:45.000 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:45.000 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:45.001 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:45.000+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:45.001 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T20:10:45.012 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T20:10:45.025 INFO:tasks.workunit.client.0.smithi039.stderr:stty: 'standard input': Inappropriate ioctl for device 2024-04-25T20:10:45.035 INFO:tasks.workunit.client.0.smithi039.stderr:+ test_rbd_journal 2024-04-25T20:10:45.035 INFO:tasks.workunit.client.0.smithi039.stderr:+ local image=testrbdjournal47106 2024-04-25T20:10:45.035 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd create --image-feature exclusive-lock --image-feature journaling --size 128 testrbdjournal47106 2024-04-25T20:10:45.067 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:45.067+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:45.159 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:45.159 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:45.159 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:45.159 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-04-25T20:10:45.159 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:45.159 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4939.0:2 1.1 1:9be44ac3:::rbd_id.testrbdjournal47106:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-04-25T20:10:45.159 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:45.101073+0000", 2024-04-25T20:10:45.159 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.058037020000000002, 2024-04-25T20:10:45.159 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00017237200000000001, 2024-04-25T20:10:45.160 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:45.160 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "reached pg", 2024-04-25T20:10:45.160 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-04-25T20:10:45.160 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4939", 2024-04-25T20:10:45.160 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1783203329", 2024-04-25T20:10:45.160 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 2 2024-04-25T20:10:45.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:45.160 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:45.160 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:45.160 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:45.160 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:45.101073+0000", 2024-04-25T20:10:45.160 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:45.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:45.160 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:45.160 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:45.161 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:45.101073+0000", 2024-04-25T20:10:45.161 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:45.161 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:45.161 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:45.161 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:45.161 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:45.101072+0000", 2024-04-25T20:10:45.161 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999981 2024-04-25T20:10:45.161 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:45.161 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:45.161 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:45.161 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:45.101079+0000", 2024-04-25T20:10:45.161 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.2899999999999997e-06 2024-04-25T20:10:45.161 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:45.161 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:45.161 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:45.162 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:45.101095+0000", 2024-04-25T20:10:45.162 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6008e-05 2024-04-25T20:10:45.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:45.162 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:45.162 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:45.162 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:45.101106+0000", 2024-04-25T20:10:45.162 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1011e-05 2024-04-25T20:10:45.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:45.162 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:45.162 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:45.162 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:45.101149+0000", 2024-04-25T20:10:45.162 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2475999999999998e-05 2024-04-25T20:10:45.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:45.162 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:45.163 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:45.163 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:45.101246+0000", 2024-04-25T20:10:45.163 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7312000000000006e-05 2024-04-25T20:10:45.163 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:45.163 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:45.163 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:45.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:45.163 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:45.163 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4939.0:3 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-04-25T20:10:45.163 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:45.103501+0000", 2024-04-25T20:10:45.163 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.055609371999999997, 2024-04-25T20:10:45.163 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00014793099999999999, 2024-04-25T20:10:45.163 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:45.163 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "reached pg", 2024-04-25T20:10:45.163 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-04-25T20:10:45.163 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4939", 2024-04-25T20:10:45.164 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1783203329", 2024-04-25T20:10:45.164 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 3 2024-04-25T20:10:45.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:45.164 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:45.164 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:45.164 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:45.164 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:45.103501+0000", 2024-04-25T20:10:45.164 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:45.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:45.164 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:45.164 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:45.164 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:45.103501+0000", 2024-04-25T20:10:45.164 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:45.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:45.164 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:45.164 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:45.165 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:45.103500+0000", 2024-04-25T20:10:45.165 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:45.165 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:45.165 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:45.165 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:45.165 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:45.103513+0000", 2024-04-25T20:10:45.165 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3397999999999999e-05 2024-04-25T20:10:45.165 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:45.165 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:45.165 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:45.165 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:45.103525+0000", 2024-04-25T20:10:45.165 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1392999999999999e-05 2024-04-25T20:10:45.165 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:45.165 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:45.165 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:45.165 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:45.103531+0000", 2024-04-25T20:10:45.166 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4969999999999998e-06 2024-04-25T20:10:45.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:45.166 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:45.166 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:45.166 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:45.103571+0000", 2024-04-25T20:10:45.166 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9643e-05 2024-04-25T20:10:45.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:45.166 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:45.166 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:45.166 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:45.103649+0000", 2024-04-25T20:10:45.166 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8010999999999996e-05 2024-04-25T20:10:45.166 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:45.166 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:45.166 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4939.0:4 1.2 1:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-04-25T20:10:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:45.103830+0000", 2024-04-25T20:10:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.055280137, 2024-04-25T20:10:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000111037, 2024-04-25T20:10:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "reached pg", 2024-04-25T20:10:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-04-25T20:10:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4939", 2024-04-25T20:10:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1783203329", 2024-04-25T20:10:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 4 2024-04-25T20:10:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:45.167 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:45.168 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:45.168 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:45.103830+0000", 2024-04-25T20:10:45.168 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:45.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:45.168 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:45.168 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:45.168 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:45.103830+0000", 2024-04-25T20:10:45.168 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:45.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:45.168 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:45.168 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:45.168 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:45.103830+0000", 2024-04-25T20:10:45.168 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:45.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:45.168 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:45.169 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:45.169 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:45.103834+0000", 2024-04-25T20:10:45.169 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2880000000000003e-06 2024-04-25T20:10:45.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:45.169 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:45.169 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:45.169 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:45.103843+0000", 2024-04-25T20:10:45.169 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.507e-06 2024-04-25T20:10:45.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:45.169 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:45.169 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:45.169 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:45.103848+0000", 2024-04-25T20:10:45.169 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0900000000000004e-06 2024-04-25T20:10:45.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:45.170 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:45.170 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:45.170 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:45.103871+0000", 2024-04-25T20:10:45.170 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3784000000000001e-05 2024-04-25T20:10:45.170 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:45.170 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:45.170 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:45.170 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:45.103941+0000", 2024-04-25T20:10:45.170 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9925000000000006e-05 2024-04-25T20:10:45.170 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:45.170 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:45.170 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:45.170 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:45.170 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:45.171 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:45.172 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T20:10:45.172 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:10:45.173 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:10:45.173 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:45.171+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:45.203 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:10:45.204 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:10:45.238 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:10:45.238 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:10:45.273 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:45.272+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:45.273 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:10:45.273 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:10:45.276 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-04-25T20:10:45.276+0000 7f249d8be640 -1 mon.a@0(leader).osd e57 definitely_dead 0 2024-04-25T20:10:45.309 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:10:45.309 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:10:45.310 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:45.310 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:45.319 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T20:10:45.340 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:10:45.340 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:10:45.357 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:10:45.357 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:10:45.373 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:45.373+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:45.445 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:45.445 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:45.454 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T20:10:45.495 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:45.495+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:45.575 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:45.574+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:45.576 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:10:45.576 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:10:45.600 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:45.601 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:45.610 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T20:10:45.694 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:45.694+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:45.757 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:45.757 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:45.766 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T20:10:45.777 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:45.777+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:45.878 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:45.878+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:45.914 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:45.914 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:45.920 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T20:10:45.953 INFO:teuthology.orchestra.run.smithi039.stderr:marked in osd.2. 2024-04-25T20:10:45.953 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is unset 2024-04-25T20:10:45.969 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-04-25T20:10:46.009 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:46.009+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:46.060 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:46.060 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:46.069 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T20:10:46.080 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:46.079+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:46.181 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:46.180+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:46.215 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:46.216 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:46.225 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T20:10:46.282 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:46.282+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:46.370 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:46.370 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:46.379 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T20:10:46.383 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:46.383+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:46.484 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:46.483+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:46.484 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:46.484 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:46.493 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T20:10:46.585 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:46.585+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:46.607 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:46.607 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:46.616 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T20:10:46.686 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:46.685+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:46.718 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:46.718 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:46.727 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T20:10:46.787 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:46.787+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:46.868 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:46.868 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:46.877 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T20:10:46.921 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:46.920+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:46.980 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:46.980 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:46.990 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T20:10:46.990 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:46.988+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:47.018 INFO:tasks.workunit.client.0.smithi039.stderr:++ rbd info testrbdjournal47106 --format=xml 2024-04-25T20:10:47.018 INFO:tasks.workunit.client.0.smithi039.stderr:++ xmlstarlet sel -t -v //image/journal 2024-04-25T20:10:47.086 INFO:tasks.workunit.client.0.smithi039.stderr:+ local journal=134bdebc9a96 2024-04-25T20:10:47.086 INFO:tasks.workunit.client.0.smithi039.stderr:+ test -n 134bdebc9a96 2024-04-25T20:10:47.087 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd journal info 134bdebc9a96 2024-04-25T20:10:47.090 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:47.089+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:47.120 INFO:tasks.workunit.client.0.smithi039.stdout:rbd journal '134bdebc9a96': 2024-04-25T20:10:47.120 INFO:tasks.workunit.client.0.smithi039.stdout: header_oid: journal.134bdebc9a96 2024-04-25T20:10:47.121 INFO:tasks.workunit.client.0.smithi039.stdout: object_oid_prefix: journal_data.1.134bdebc9a96. 2024-04-25T20:10:47.121 INFO:tasks.workunit.client.0.smithi039.stdout: order: 24 (16 MiB objects) 2024-04-25T20:10:47.121 INFO:tasks.workunit.client.0.smithi039.stdout: splay_width: 4 2024-04-25T20:10:47.123 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd journal info --journal 134bdebc9a96 2024-04-25T20:10:47.133 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:47.133 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:47.142 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T20:10:47.157 INFO:tasks.workunit.client.0.smithi039.stdout:rbd journal '134bdebc9a96': 2024-04-25T20:10:47.157 INFO:tasks.workunit.client.0.smithi039.stdout: header_oid: journal.134bdebc9a96 2024-04-25T20:10:47.157 INFO:tasks.workunit.client.0.smithi039.stdout: object_oid_prefix: journal_data.1.134bdebc9a96. 2024-04-25T20:10:47.157 INFO:tasks.workunit.client.0.smithi039.stdout: order: 24 (16 MiB objects) 2024-04-25T20:10:47.157 INFO:tasks.workunit.client.0.smithi039.stdout: splay_width: 4 2024-04-25T20:10:47.160 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd journal info --image testrbdjournal47106 2024-04-25T20:10:47.190 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:47.190+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:47.206 INFO:tasks.workunit.client.0.smithi039.stdout:rbd journal '134bdebc9a96': 2024-04-25T20:10:47.206 INFO:tasks.workunit.client.0.smithi039.stdout: header_oid: journal.134bdebc9a96 2024-04-25T20:10:47.206 INFO:tasks.workunit.client.0.smithi039.stdout: object_oid_prefix: journal_data.1.134bdebc9a96. 2024-04-25T20:10:47.206 INFO:tasks.workunit.client.0.smithi039.stdout: order: 24 (16 MiB objects) 2024-04-25T20:10:47.207 INFO:tasks.workunit.client.0.smithi039.stdout: splay_width: 4 2024-04-25T20:10:47.209 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd feature disable testrbdjournal47106 journaling 2024-04-25T20:10:47.271 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd info testrbdjournal47106 --format=xml 2024-04-25T20:10:47.271 INFO:tasks.workunit.client.0.smithi039.stderr:+ expect_false xmlstarlet sel -t -v //image/journal 2024-04-25T20:10:47.271 INFO:tasks.workunit.client.0.smithi039.stderr:+ set -x 2024-04-25T20:10:47.271 INFO:tasks.workunit.client.0.smithi039.stderr:+ xmlstarlet sel -t -v //image/journal 2024-04-25T20:10:47.289 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:47.289 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:47.291 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:47.291+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:47.298 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T20:10:47.314 INFO:tasks.workunit.client.0.smithi039.stderr:+ return 0 2024-04-25T20:10:47.314 INFO:tasks.workunit.client.0.smithi039.stderr:+ expect_false rbd journal info 134bdebc9a96 2024-04-25T20:10:47.314 INFO:tasks.workunit.client.0.smithi039.stderr:+ set -x 2024-04-25T20:10:47.314 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd journal info 134bdebc9a96 2024-04-25T20:10:47.346 INFO:tasks.workunit.client.0.smithi039.stderr:failed to get journal metadata: (2) No such file or directory 2024-04-25T20:10:47.346 INFO:tasks.workunit.client.0.smithi039.stderr:rbd: journal info: (2) No such file or directory 2024-04-25T20:10:47.349 INFO:tasks.workunit.client.0.smithi039.stderr:+ return 0 2024-04-25T20:10:47.349 INFO:tasks.workunit.client.0.smithi039.stderr:+ expect_false rbd journal info --image testrbdjournal47106 2024-04-25T20:10:47.349 INFO:tasks.workunit.client.0.smithi039.stderr:+ set -x 2024-04-25T20:10:47.349 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd journal info --image testrbdjournal47106 2024-04-25T20:10:47.387 INFO:tasks.workunit.client.0.smithi039.stderr:rbd: journaling is not enabled for image testrbdjournal47106 2024-04-25T20:10:47.390 INFO:tasks.workunit.client.0.smithi039.stderr:+ return 0 2024-04-25T20:10:47.390 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd feature enable testrbdjournal47106 journaling 2024-04-25T20:10:47.391 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:47.391+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:47.401 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:47.401 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:47.410 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T20:10:47.452 INFO:tasks.workunit.client.0.smithi039.stderr:++ rbd info testrbdjournal47106 --format=xml 2024-04-25T20:10:47.452 INFO:tasks.workunit.client.0.smithi039.stderr:++ xmlstarlet sel -t -v //image/journal 2024-04-25T20:10:47.494 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:47.493+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:47.496 INFO:tasks.workunit.client.0.smithi039.stderr:+ local journal1=134bdebc9a96 2024-04-25T20:10:47.496 INFO:tasks.workunit.client.0.smithi039.stderr:+ test 134bdebc9a96 = 134bdebc9a96 2024-04-25T20:10:47.496 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd journal info 134bdebc9a96 2024-04-25T20:10:47.527 INFO:tasks.workunit.client.0.smithi039.stdout:rbd journal '134bdebc9a96': 2024-04-25T20:10:47.527 INFO:tasks.workunit.client.0.smithi039.stdout: header_oid: journal.134bdebc9a96 2024-04-25T20:10:47.527 INFO:tasks.workunit.client.0.smithi039.stdout: object_oid_prefix: journal_data.1.134bdebc9a96. 2024-04-25T20:10:47.528 INFO:tasks.workunit.client.0.smithi039.stdout: order: 24 (16 MiB objects) 2024-04-25T20:10:47.528 INFO:tasks.workunit.client.0.smithi039.stdout: splay_width: 4 2024-04-25T20:10:47.530 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd journal status 134bdebc9a96 2024-04-25T20:10:47.552 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:47.552 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:47.561 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T20:10:47.562 INFO:tasks.workunit.client.0.smithi039.stdout:minimum_set: 0 2024-04-25T20:10:47.562 INFO:tasks.workunit.client.0.smithi039.stdout:active_set: 0 2024-04-25T20:10:47.563 INFO:tasks.workunit.client.0.smithi039.stdout:registered clients: 2024-04-25T20:10:47.563 INFO:tasks.workunit.client.0.smithi039.stdout: [id=, commit_position=[positions=[]], state=connected] 2024-04-25T20:10:47.566 INFO:tasks.workunit.client.0.smithi039.stderr:+ local count=10 2024-04-25T20:10:47.566 INFO:tasks.workunit.client.0.smithi039.stderr:+ save_commit_position 134bdebc9a96 2024-04-25T20:10:47.566 INFO:tasks.workunit.client.0.smithi039.stderr:+ local journal=134bdebc9a96 2024-04-25T20:10:47.566 INFO:tasks.workunit.client.0.smithi039.stderr:+ rados -p rbd getomapval journal.134bdebc9a96 client_ /tmp/rbd_journal47106/134bdebc9a96.client_.omap 2024-04-25T20:10:47.587 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T20:10:47.601 INFO:tasks.workunit.client.0.smithi039.stderr:Writing to /tmp/rbd_journal47106/134bdebc9a96.client_.omap 2024-04-25T20:10:47.601 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:47.601+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:47.603 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd bench --io-type write testrbdjournal47106 --io-size 4096 --io-threads 1 --io-total 40960 --io-pattern seq 2024-04-25T20:10:47.664 INFO:tasks.workunit.client.0.smithi039.stdout:bench type write io_size 4096 io_threads 1 bytes 40960 pattern sequential 2024-04-25T20:10:47.677 INFO:tasks.workunit.client.0.smithi039.stdout: SEC OPS OPS/SEC BYTES/SEC 2024-04-25T20:10:47.677 INFO:tasks.workunit.client.0.smithi039.stdout:elapsed: 0 ops: 10 ops/sec: 769.246 bytes/sec: 3.0 MiB/s 2024-04-25T20:10:47.689 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd journal status --image testrbdjournal47106 2024-04-25T20:10:47.689 INFO:tasks.workunit.client.0.smithi039.stderr:+ fgrep tid=9 2024-04-25T20:10:47.693 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:47.693+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:47.710 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:47.710 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:47.716 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:10:47.716 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:10:47.719 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T20:10:47.736 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:10:47.737 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:10:47.743 INFO:tasks.workunit.client.0.smithi039.stdout: [id=, commit_position=[positions=[[object_number=3, tag_tid=1, entry_tid=11], [object_number=2, tag_tid=1, entry_tid=10], [object_number=1, tag_tid=1, entry_tid=9], [object_number=0, tag_tid=1, entry_tid=8]]], state=connected] 2024-04-25T20:10:47.743 INFO:tasks.workunit.client.0.smithi039.stderr:+ restore_commit_position 134bdebc9a96 2024-04-25T20:10:47.743 INFO:tasks.workunit.client.0.smithi039.stderr:+ local journal=134bdebc9a96 2024-04-25T20:10:47.743 INFO:tasks.workunit.client.0.smithi039.stderr:+ rados -p rbd setomapval journal.134bdebc9a96 client_ 2024-04-25T20:10:47.753 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:10:47.754 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:10:47.772 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:10:47.773 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:10:47.784 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd journal status --image testrbdjournal47106 2024-04-25T20:10:47.784 INFO:tasks.workunit.client.0.smithi039.stderr:+ fgrep 'positions=[]' 2024-04-25T20:10:47.793 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:10:47.793 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:10:47.794 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:47.794+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:47.810 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:10:47.810 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:10:47.823 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:47.824 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:47.829 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:10:47.829 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:10:47.833 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T20:10:47.834 INFO:tasks.workunit.client.0.smithi039.stdout: [id=, commit_position=[positions=[]], state=connected] 2024-04-25T20:10:47.835 INFO:tasks.workunit.client.0.smithi039.stderr:++ rbd journal inspect --verbose 134bdebc9a96 2024-04-25T20:10:47.835 INFO:tasks.workunit.client.0.smithi039.stderr:++ grep -c 'event_type.*AioWrite' 2024-04-25T20:10:47.846 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:10:47.846 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:10:47.880 INFO:tasks.workunit.client.0.smithi039.stderr:2024-04-25T20:10:47.880+0000 7f03fcc10640 -1 JournalMetadata: 0x55d719756750 failed to locate client: INSPECT 2024-04-25T20:10:47.884 INFO:tasks.workunit.client.0.smithi039.stderr:+ local count1=10 2024-04-25T20:10:47.884 INFO:tasks.workunit.client.0.smithi039.stderr:+ test 10 -eq 10 2024-04-25T20:10:47.884 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd journal export 134bdebc9a96 /tmp/rbd_journal47106/journal.export 2024-04-25T20:10:47.895 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:47.894+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:47.923 INFO:tasks.workunit.client.0.smithi039.stdout:12 entries processed, 0 errors 2024-04-25T20:10:47.925 INFO:tasks.workunit.client.0.smithi039.stderr:2024-04-25T20:10:47.925+0000 7fd95dd90640 -1 JournalMetadata: 0x560c500f2210 failed to locate client: EXPORT 2024-04-25T20:10:47.929 INFO:tasks.workunit.client.0.smithi039.stderr:++ stat -c %s /tmp/rbd_journal47106/journal.export 2024-04-25T20:10:47.930 INFO:tasks.workunit.client.0.smithi039.stderr:+ local size=55926 2024-04-25T20:10:47.930 INFO:tasks.workunit.client.0.smithi039.stderr:+ test 55926 -gt 0 2024-04-25T20:10:47.930 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd export testrbdjournal47106 /tmp/rbd_journal47106/testrbdjournal47106.export 2024-04-25T20:10:47.936 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:47.936 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:47.937 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:47.937 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:47.937 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:47.946 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T20:10:47.968 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-25T20:10:47.996 INFO:tasks.workunit.client.0.smithi039.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-25T20:10:47.996 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:47.996+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:48.001 INFO:tasks.workunit.client.0.smithi039.stderr:+ local image1=testrbdjournal471061 2024-04-25T20:10:48.001 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd create --image-feature exclusive-lock --image-feature journaling --size 128 testrbdjournal471061 2024-04-25T20:10:48.056 INFO:tasks.workunit.client.0.smithi039.stderr:++ rbd info testrbdjournal471061 --format=xml 2024-04-25T20:10:48.056 INFO:tasks.workunit.client.0.smithi039.stderr:++ xmlstarlet sel -t -v //image/journal 2024-04-25T20:10:48.092 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:48.092 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:48.092 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:48.092 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [ 2024-04-25T20:10:48.093 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.093 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:48.093 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:10:48.093 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 91.308632408999998, 2024-04-25T20:10:48.093 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.34154860100000001, 2024-04-25T20:10:48.093 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:48.093 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.093 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:48.093 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:48.093 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:48.093 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 1 2024-04-25T20:10:48.093 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.093 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:48.093 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.094 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:48.094 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:10:48.094 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.094 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.094 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.094 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:48.094 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782262+0000", 2024-04-25T20:10:48.094 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.094 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.094 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.094 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:48.094 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782261+0000", 2024-04-25T20:10:48.094 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999986 2024-04-25T20:10:48.094 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.094 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.095 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:48.095 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782268+0000", 2024-04-25T20:10:48.095 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.8140000000000001e-06 2024-04-25T20:10:48.095 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.095 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.095 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:48.095 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782279+0000", 2024-04-25T20:10:48.095 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0665999999999999e-05 2024-04-25T20:10:48.095 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.095 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.095 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.095 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782290+0000", 2024-04-25T20:10:48.095 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.1343e-05 2024-04-25T20:10:48.095 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.095 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.095 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:48.096 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782340+0000", 2024-04-25T20:10:48.096 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.9311999999999998e-05 2024-04-25T20:10:48.096 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.096 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.096 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for peered", 2024-04-25T20:10:48.096 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:16.782348+0000", 2024-04-25T20:10:48.096 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.0690000000000002e-06 2024-04-25T20:10:48.096 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.096 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.096 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:48.096 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120520+0000", 2024-04-25T20:10:48.096 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.33817188100000001 2024-04-25T20:10:48.096 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.096 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.096 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:48.096 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120656+0000", 2024-04-25T20:10:48.097 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000135897 2024-04-25T20:10:48.097 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.097 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.097 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:48.097 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.120802+0000", 2024-04-25T20:10:48.097 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00014691199999999999 2024-04-25T20:10:48.097 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.097 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.097 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:48.097 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.122368+0000", 2024-04-25T20:10:48.097 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0015651040000000001 2024-04-25T20:10:48.097 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.097 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.097 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:48.097 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123729+0000", 2024-04-25T20:10:48.098 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0013610499999999999 2024-04-25T20:10:48.098 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.098 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.098 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:48.098 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123769+0000", 2024-04-25T20:10:48.098 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0719000000000001e-05 2024-04-25T20:10:48.098 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.098 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.098 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:48.098 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.123811+0000", 2024-04-25T20:10:48.098 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1267999999999999e-05 2024-04-25T20:10:48.098 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.098 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:48.098 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.099 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.099 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.099 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:67 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:48.099 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:10:48.099 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 90.879213522000001, 2024-04-25T20:10:48.099 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.006947651, 2024-04-25T20:10:48.099 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:48.099 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.099 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:48.099 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:48.099 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:48.099 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 67 2024-04-25T20:10:48.099 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.099 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:48.100 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.100 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:48.100 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:10:48.100 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.100 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.100 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.100 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:48.100 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211681+0000", 2024-04-25T20:10:48.100 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.100 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.100 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.100 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:48.100 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211680+0000", 2024-04-25T20:10:48.100 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:48.100 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.101 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.101 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:48.101 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211684+0000", 2024-04-25T20:10:48.101 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4249999999999998e-06 2024-04-25T20:10:48.101 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.101 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.101 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:48.101 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211690+0000", 2024-04-25T20:10:48.101 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.7360000000000004e-06 2024-04-25T20:10:48.101 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.101 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.101 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.101 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211694+0000", 2024-04-25T20:10:48.101 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4860000000000002e-06 2024-04-25T20:10:48.101 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.102 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.102 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:48.102 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211727+0000", 2024-04-25T20:10:48.102 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3445999999999998e-05 2024-04-25T20:10:48.102 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.102 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.102 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:48.102 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211778+0000", 2024-04-25T20:10:48.102 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1050000000000001e-05 2024-04-25T20:10:48.102 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.102 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.102 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:48.102 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211851+0000", 2024-04-25T20:10:48.102 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2914e-05 2024-04-25T20:10:48.102 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.103 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.103 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:48.103 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213824+0000", 2024-04-25T20:10:48.103 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0019726520000000001 2024-04-25T20:10:48.103 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.103 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.103 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:48.103 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218579+0000", 2024-04-25T20:10:48.103 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0047553159999999999 2024-04-25T20:10:48.103 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.103 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.103 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:48.103 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218607+0000", 2024-04-25T20:10:48.103 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8144999999999999e-05 2024-04-25T20:10:48.103 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.104 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.104 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:48.104 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.218629+0000", 2024-04-25T20:10:48.104 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.1044e-05 2024-04-25T20:10:48.104 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.104 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:48.104 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.104 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.104 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.104 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:68 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:48.104 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:48.104 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 90.879195181, 2024-04-25T20:10:48.104 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012255596000000001, 2024-04-25T20:10:48.104 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:48.104 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.104 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:48.104 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:48.105 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:48.105 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 68 2024-04-25T20:10:48.105 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.105 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:48.105 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.105 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:48.105 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:48.105 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.105 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.105 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.105 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:48.105 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:48.105 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.105 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.105 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.106 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:48.106 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211699+0000", 2024-04-25T20:10:48.106 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:48.106 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.106 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.106 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:48.106 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211701+0000", 2024-04-25T20:10:48.106 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2029999999999999e-06 2024-04-25T20:10:48.106 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.106 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.106 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:48.106 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211705+0000", 2024-04-25T20:10:48.106 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.2320000000000003e-06 2024-04-25T20:10:48.106 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.107 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.107 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.107 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211708+0000", 2024-04-25T20:10:48.107 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.6290000000000002e-06 2024-04-25T20:10:48.107 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.107 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.107 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:48.107 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212136+0000", 2024-04-25T20:10:48.107 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00042764099999999998 2024-04-25T20:10:48.107 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.107 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.107 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:48.107 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212212+0000", 2024-04-25T20:10:48.107 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.6315000000000004e-05 2024-04-25T20:10:48.107 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.107 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.107 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:48.108 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212306+0000", 2024-04-25T20:10:48.108 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.3556999999999999e-05 2024-04-25T20:10:48.108 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.108 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.108 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:48.108 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213829+0000", 2024-04-25T20:10:48.108 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001523256 2024-04-25T20:10:48.108 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.108 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.108 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:48.108 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223904+0000", 2024-04-25T20:10:48.108 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010075387 2024-04-25T20:10:48.108 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.108 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.108 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:48.109 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223934+0000", 2024-04-25T20:10:48.109 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9988000000000002e-05 2024-04-25T20:10:48.109 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.109 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.109 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:48.109 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223955+0000", 2024-04-25T20:10:48.109 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0630999999999999e-05 2024-04-25T20:10:48.109 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.109 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:48.109 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.109 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.109 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.109 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:69 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:48.109 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:48.109 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 90.879173445999996, 2024-04-25T20:10:48.109 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012316863000000001, 2024-04-25T20:10:48.110 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:48.110 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.110 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:48.110 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:48.110 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:48.110 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 69 2024-04-25T20:10:48.110 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.110 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:48.110 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.110 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:48.110 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:48.110 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.110 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.110 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.111 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:48.111 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:48.111 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.111 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.111 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.111 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:48.111 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211721+0000", 2024-04-25T20:10:48.111 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:48.111 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.111 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.111 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:48.111 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211762+0000", 2024-04-25T20:10:48.111 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1703999999999999e-05 2024-04-25T20:10:48.111 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.111 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.111 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:48.111 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211768+0000", 2024-04-25T20:10:48.112 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.3430000000000003e-06 2024-04-25T20:10:48.112 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.112 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.112 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.112 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211771+0000", 2024-04-25T20:10:48.112 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.18e-06 2024-04-25T20:10:48.112 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.112 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.112 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:48.112 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212652+0000", 2024-04-25T20:10:48.112 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00088095800000000004 2024-04-25T20:10:48.112 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.112 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.112 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:48.112 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212703+0000", 2024-04-25T20:10:48.112 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.0655999999999997e-05 2024-04-25T20:10:48.113 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.113 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.113 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:48.113 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.212781+0000", 2024-04-25T20:10:48.113 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.8852999999999999e-05 2024-04-25T20:10:48.113 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.113 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.113 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:48.113 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.214131+0000", 2024-04-25T20:10:48.113 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0013496039999999999 2024-04-25T20:10:48.113 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.113 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.113 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:48.113 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.223986+0000", 2024-04-25T20:10:48.113 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.009854804 2024-04-25T20:10:48.113 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.113 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.114 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:48.114 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.224017+0000", 2024-04-25T20:10:48.114 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.1068999999999997e-05 2024-04-25T20:10:48.114 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.114 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.114 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:48.114 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.224038+0000", 2024-04-25T20:10:48.114 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0965e-05 2024-04-25T20:10:48.114 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.114 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:48.114 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.114 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.114 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.114 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:70 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-25T20:10:48.114 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:48.115 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 90.879118364999997, 2024-04-25T20:10:48.115 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.015776348999999999, 2024-04-25T20:10:48.115 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:48.115 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.115 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:48.115 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:48.115 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:48.115 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 70 2024-04-25T20:10:48.115 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.115 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:48.115 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.115 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:48.115 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:48.115 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.115 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.115 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.115 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:48.116 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:48.116 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.116 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.116 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.116 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:48.116 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211776+0000", 2024-04-25T20:10:48.116 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:48.116 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.116 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.116 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:48.116 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211778+0000", 2024-04-25T20:10:48.116 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5330000000000001e-06 2024-04-25T20:10:48.116 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.116 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.116 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:48.116 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211782+0000", 2024-04-25T20:10:48.117 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4079999999999998e-06 2024-04-25T20:10:48.117 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.117 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.117 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.117 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.211786+0000", 2024-04-25T20:10:48.117 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.6190000000000002e-06 2024-04-25T20:10:48.117 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.117 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.117 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:48.117 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213374+0000", 2024-04-25T20:10:48.117 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0015881859999999999 2024-04-25T20:10:48.117 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.117 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.117 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:48.117 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213478+0000", 2024-04-25T20:10:48.117 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000103464 2024-04-25T20:10:48.118 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.118 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.118 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:48.118 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.213601+0000", 2024-04-25T20:10:48.118 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00012324300000000001 2024-04-25T20:10:48.118 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.118 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.118 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:48.118 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.214788+0000", 2024-04-25T20:10:48.118 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011868569999999999 2024-04-25T20:10:48.118 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.118 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.118 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:48.118 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227504+0000", 2024-04-25T20:10:48.118 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.012715749 2024-04-25T20:10:48.118 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.119 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.119 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:48.119 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227532+0000", 2024-04-25T20:10:48.119 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.7937e-05 2024-04-25T20:10:48.119 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.119 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.119 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:48.119 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:17.227552+0000", 2024-04-25T20:10:48.119 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.0772999999999999e-05 2024-04-25T20:10:48.119 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.119 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:48.119 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.119 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.119 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.119 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:131 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:48.120 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:10:48.120 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 89.970108519999997, 2024-04-25T20:10:48.120 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0086158810000000006, 2024-04-25T20:10:48.120 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:48.120 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.120 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:48.120 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:48.120 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:48.120 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 131 2024-04-25T20:10:48.120 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.120 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:48.120 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.120 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:48.120 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:10:48.120 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.120 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.120 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.121 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:48.121 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120786+0000", 2024-04-25T20:10:48.121 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.121 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.121 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.121 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:48.121 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120785+0000", 2024-04-25T20:10:48.121 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:48.121 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.121 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.121 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:48.121 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120880+0000", 2024-04-25T20:10:48.121 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.4925000000000003e-05 2024-04-25T20:10:48.121 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.121 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.121 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:48.122 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120891+0000", 2024-04-25T20:10:48.122 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0915e-05 2024-04-25T20:10:48.122 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.122 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.122 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.122 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120895+0000", 2024-04-25T20:10:48.122 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.8070000000000001e-06 2024-04-25T20:10:48.122 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.122 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.122 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:48.122 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121640+0000", 2024-04-25T20:10:48.122 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00074468500000000003 2024-04-25T20:10:48.122 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.122 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.122 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:48.123 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121684+0000", 2024-04-25T20:10:48.123 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4452000000000001e-05 2024-04-25T20:10:48.123 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.123 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.123 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:48.123 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121753+0000", 2024-04-25T20:10:48.123 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8508000000000002e-05 2024-04-25T20:10:48.123 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.123 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.123 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:48.123 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123618+0000", 2024-04-25T20:10:48.123 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0018655099999999999 2024-04-25T20:10:48.123 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.123 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.123 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:48.124 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129248+0000", 2024-04-25T20:10:48.124 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0056295540000000002 2024-04-25T20:10:48.124 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.124 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.124 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:48.124 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129368+0000", 2024-04-25T20:10:48.124 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000120244 2024-04-25T20:10:48.124 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.124 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.124 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:48.124 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129402+0000", 2024-04-25T20:10:48.124 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3812000000000001e-05 2024-04-25T20:10:48.124 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.124 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:48.124 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.124 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.125 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.125 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:132 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:48.125 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:48.125 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 89.969994310000004, 2024-04-25T20:10:48.125 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0085898169999999996, 2024-04-25T20:10:48.125 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:48.125 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.125 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:48.125 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:48.125 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:48.125 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 132 2024-04-25T20:10:48.125 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.125 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:48.125 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.125 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:48.125 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:48.125 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.126 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.126 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.126 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:48.126 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:48.126 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.126 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.126 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.126 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:48.126 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120900+0000", 2024-04-25T20:10:48.126 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:48.126 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.126 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.126 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:48.127 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120902+0000", 2024-04-25T20:10:48.127 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.6129999999999998e-06 2024-04-25T20:10:48.127 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.127 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.127 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:48.127 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120906+0000", 2024-04-25T20:10:48.127 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6569999999999999e-06 2024-04-25T20:10:48.127 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.127 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.127 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.127 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120909+0000", 2024-04-25T20:10:48.127 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5550000000000001e-06 2024-04-25T20:10:48.127 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.127 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.128 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:48.128 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122098+0000", 2024-04-25T20:10:48.128 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011898049999999999 2024-04-25T20:10:48.128 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.128 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.128 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:48.128 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122172+0000", 2024-04-25T20:10:48.128 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3917000000000004e-05 2024-04-25T20:10:48.128 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.128 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.128 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:48.128 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122288+0000", 2024-04-25T20:10:48.128 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00011566 2024-04-25T20:10:48.128 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.128 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.129 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:48.129 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124519+0000", 2024-04-25T20:10:48.129 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0022308359999999999 2024-04-25T20:10:48.129 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.129 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.129 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:48.129 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129414+0000", 2024-04-25T20:10:48.129 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048953720000000003 2024-04-25T20:10:48.129 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.129 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.129 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:48.129 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129458+0000", 2024-04-25T20:10:48.129 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.3426000000000001e-05 2024-04-25T20:10:48.129 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.129 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.129 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:48.130 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129490+0000", 2024-04-25T20:10:48.130 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2329999999999997e-05 2024-04-25T20:10:48.130 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.130 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:48.130 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.130 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.130 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.130 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:133 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:48.130 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:48.130 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 89.969940441000006, 2024-04-25T20:10:48.130 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0086066270000000004, 2024-04-25T20:10:48.130 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:48.130 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.130 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:48.130 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:48.130 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:48.131 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 133 2024-04-25T20:10:48.131 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.131 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:48.131 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.131 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:48.131 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:48.131 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.131 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.131 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.131 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:48.131 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:48.131 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.131 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.131 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.131 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:48.132 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120954+0000", 2024-04-25T20:10:48.132 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:48.132 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.132 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.132 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:48.132 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120956+0000", 2024-04-25T20:10:48.132 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5289999999999998e-06 2024-04-25T20:10:48.132 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.132 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.132 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:48.132 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120962+0000", 2024-04-25T20:10:48.132 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.9490000000000001e-06 2024-04-25T20:10:48.132 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.132 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.132 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.132 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.120965+0000", 2024-04-25T20:10:48.132 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.1860000000000001e-06 2024-04-25T20:10:48.133 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.133 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.133 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:48.133 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122666+0000", 2024-04-25T20:10:48.133 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0017004500000000001 2024-04-25T20:10:48.133 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.133 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.133 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:48.133 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122741+0000", 2024-04-25T20:10:48.133 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.5297999999999996e-05 2024-04-25T20:10:48.133 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.133 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.133 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:48.133 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.122858+0000", 2024-04-25T20:10:48.133 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00011723899999999999 2024-04-25T20:10:48.133 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.134 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.134 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:48.134 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124625+0000", 2024-04-25T20:10:48.134 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0017670100000000001 2024-04-25T20:10:48.134 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.134 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.134 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:48.134 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129497+0000", 2024-04-25T20:10:48.134 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048713580000000001 2024-04-25T20:10:48.134 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.134 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.134 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:48.134 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129532+0000", 2024-04-25T20:10:48.134 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5704999999999998e-05 2024-04-25T20:10:48.134 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.135 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.135 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:48.135 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129561+0000", 2024-04-25T20:10:48.135 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8283e-05 2024-04-25T20:10:48.135 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.135 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:48.135 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.135 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.135 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.135 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:134 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:48.135 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:48.135 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 89.969838242999998, 2024-04-25T20:10:48.135 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0085771179999999999, 2024-04-25T20:10:48.135 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:48.135 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.135 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:48.136 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:48.136 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:48.136 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 134 2024-04-25T20:10:48.136 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.136 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:48.136 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.136 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:48.136 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:48.136 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.136 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.136 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.136 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:48.136 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:48.136 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.136 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.136 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.136 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:48.137 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121056+0000", 2024-04-25T20:10:48.137 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:48.137 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.137 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.137 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:48.137 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121157+0000", 2024-04-25T20:10:48.137 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00010104599999999999 2024-04-25T20:10:48.137 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.137 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.137 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:48.137 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121189+0000", 2024-04-25T20:10:48.137 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2589000000000003e-05 2024-04-25T20:10:48.137 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.137 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.137 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.138 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121195+0000", 2024-04-25T20:10:48.138 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.083e-06 2024-04-25T20:10:48.138 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.138 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.138 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:48.138 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123136+0000", 2024-04-25T20:10:48.138 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0019409780000000001 2024-04-25T20:10:48.138 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.138 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.138 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:48.138 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123181+0000", 2024-04-25T20:10:48.138 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4267999999999997e-05 2024-04-25T20:10:48.138 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.138 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.139 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:48.139 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123247+0000", 2024-04-25T20:10:48.139 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.6086000000000004e-05 2024-04-25T20:10:48.139 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.139 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.139 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:48.139 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125639+0000", 2024-04-25T20:10:48.139 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.002392 2024-04-25T20:10:48.139 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.139 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.139 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:48.139 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129567+0000", 2024-04-25T20:10:48.139 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0039282170000000003 2024-04-25T20:10:48.140 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.140 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.140 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:48.140 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129605+0000", 2024-04-25T20:10:48.140 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.7660999999999997e-05 2024-04-25T20:10:48.140 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.140 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.140 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:48.140 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.129633+0000", 2024-04-25T20:10:48.140 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8649999999999998e-05 2024-04-25T20:10:48.140 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.140 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:48.140 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.140 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.141 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.141 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:135 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:48.141 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:10:48.141 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 89.969679236999994, 2024-04-25T20:10:48.141 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010898453000000001, 2024-04-25T20:10:48.141 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:48.141 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.141 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:48.141 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:48.141 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:48.141 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 135 2024-04-25T20:10:48.141 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.142 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:48.142 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.142 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:48.142 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:10:48.142 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.142 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.142 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.142 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:48.142 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121215+0000", 2024-04-25T20:10:48.142 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.142 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.142 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.142 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:48.142 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121214+0000", 2024-04-25T20:10:48.142 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:48.143 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.143 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.143 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:48.143 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121218+0000", 2024-04-25T20:10:48.143 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6619999999999998e-06 2024-04-25T20:10:48.143 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.143 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.143 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:48.143 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121223+0000", 2024-04-25T20:10:48.143 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1070000000000004e-06 2024-04-25T20:10:48.143 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.143 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.143 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.143 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121227+0000", 2024-04-25T20:10:48.144 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.2820000000000001e-06 2024-04-25T20:10:48.144 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.144 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.144 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:48.144 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123649+0000", 2024-04-25T20:10:48.144 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0024221970000000001 2024-04-25T20:10:48.144 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.144 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.144 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:48.144 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123723+0000", 2024-04-25T20:10:48.144 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.4386000000000002e-05 2024-04-25T20:10:48.144 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.144 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.145 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:48.145 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.123844+0000", 2024-04-25T20:10:48.145 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000121249 2024-04-25T20:10:48.145 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.145 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.145 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:48.145 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125670+0000", 2024-04-25T20:10:48.145 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001825901 2024-04-25T20:10:48.145 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.145 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.145 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:48.145 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132043+0000", 2024-04-25T20:10:48.145 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0063724539999999996 2024-04-25T20:10:48.146 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.146 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.146 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:48.146 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132084+0000", 2024-04-25T20:10:48.146 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0963000000000001e-05 2024-04-25T20:10:48.146 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.146 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.146 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:48.146 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132114+0000", 2024-04-25T20:10:48.146 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9927e-05 2024-04-25T20:10:48.146 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.146 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:48.146 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.146 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.147 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.147 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:136 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:48.147 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:10:48.147 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 89.969663710999995, 2024-04-25T20:10:48.147 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010952513000000001, 2024-04-25T20:10:48.147 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:48.147 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.147 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:48.147 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:48.147 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:48.147 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 136 2024-04-25T20:10:48.147 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.147 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:48.147 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.147 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:48.148 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:10:48.148 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.148 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.148 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.148 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:48.148 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121231+0000", 2024-04-25T20:10:48.148 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.148 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.148 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.148 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:48.148 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121230+0000", 2024-04-25T20:10:48.148 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:48.148 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.148 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.148 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:48.148 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121236+0000", 2024-04-25T20:10:48.149 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.7320000000000003e-06 2024-04-25T20:10:48.149 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.149 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.149 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:48.149 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121242+0000", 2024-04-25T20:10:48.149 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.7810000000000002e-06 2024-04-25T20:10:48.149 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.149 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.149 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.149 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121246+0000", 2024-04-25T20:10:48.149 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.639e-06 2024-04-25T20:10:48.149 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.149 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.149 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:48.150 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124150+0000", 2024-04-25T20:10:48.150 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.002903816 2024-04-25T20:10:48.150 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.150 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.150 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:48.150 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124195+0000", 2024-04-25T20:10:48.150 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.5042e-05 2024-04-25T20:10:48.150 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.150 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.150 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:48.150 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124266+0000", 2024-04-25T20:10:48.150 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0234999999999994e-05 2024-04-25T20:10:48.150 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.150 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.150 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:48.151 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125711+0000", 2024-04-25T20:10:48.151 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0014458800000000001 2024-04-25T20:10:48.151 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.151 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.151 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:48.151 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132120+0000", 2024-04-25T20:10:48.151 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0064084399999999996 2024-04-25T20:10:48.151 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.151 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.151 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:48.151 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132155+0000", 2024-04-25T20:10:48.151 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4938000000000003e-05 2024-04-25T20:10:48.151 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.151 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.151 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:48.152 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132183+0000", 2024-04-25T20:10:48.152 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8360999999999998e-05 2024-04-25T20:10:48.152 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.152 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:48.152 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.152 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.152 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.152 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:48.152 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:48.152 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 89.969613979000002, 2024-04-25T20:10:48.152 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.01097684, 2024-04-25T20:10:48.152 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:48.152 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.152 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:48.153 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:48.153 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:48.153 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 137 2024-04-25T20:10:48.153 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.153 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:48.153 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.153 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:48.153 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:48.153 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.153 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.153 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.153 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:48.153 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:48.153 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.154 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.154 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.154 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:48.154 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121280+0000", 2024-04-25T20:10:48.154 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:48.154 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.154 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.154 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:48.154 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121328+0000", 2024-04-25T20:10:48.154 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.7908999999999998e-05 2024-04-25T20:10:48.154 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.154 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.154 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:48.154 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121337+0000", 2024-04-25T20:10:48.155 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.9369999999999993e-06 2024-04-25T20:10:48.155 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.155 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.155 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.155 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121343+0000", 2024-04-25T20:10:48.155 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.8089999999999998e-06 2024-04-25T20:10:48.155 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.155 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.155 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:48.155 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124656+0000", 2024-04-25T20:10:48.155 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0033133839999999999 2024-04-25T20:10:48.155 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.155 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.155 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:48.156 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124730+0000", 2024-04-25T20:10:48.156 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3565000000000005e-05 2024-04-25T20:10:48.156 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.156 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.156 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:48.156 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.124846+0000", 2024-04-25T20:10:48.156 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000116416 2024-04-25T20:10:48.156 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.156 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.156 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:48.156 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126756+0000", 2024-04-25T20:10:48.156 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001909991 2024-04-25T20:10:48.156 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.156 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.157 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:48.157 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132190+0000", 2024-04-25T20:10:48.157 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0054343880000000001 2024-04-25T20:10:48.157 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.157 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.157 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:48.157 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132227+0000", 2024-04-25T20:10:48.157 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6875000000000002e-05 2024-04-25T20:10:48.157 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.157 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.157 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:48.157 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132257+0000", 2024-04-25T20:10:48.157 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.0046e-05 2024-04-25T20:10:48.157 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.157 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:48.158 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.158 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.158 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.158 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:48.158 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:48.158 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 89.969544021000004, 2024-04-25T20:10:48.158 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.010978653999999999, 2024-04-25T20:10:48.158 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:48.158 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.158 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:48.158 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:48.158 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:48.158 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 138 2024-04-25T20:10:48.159 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.159 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:48.159 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.159 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:48.159 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:48.159 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.159 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.159 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.159 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:48.159 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:48.159 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.159 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.159 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.159 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:48.160 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121350+0000", 2024-04-25T20:10:48.160 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:48.160 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.160 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.160 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:48.160 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121355+0000", 2024-04-25T20:10:48.160 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.9869999999999999e-06 2024-04-25T20:10:48.160 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.160 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.160 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:48.160 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121361+0000", 2024-04-25T20:10:48.160 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.4620000000000001e-06 2024-04-25T20:10:48.160 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.160 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.161 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.161 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121373+0000", 2024-04-25T20:10:48.161 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.2035000000000001e-05 2024-04-25T20:10:48.161 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.161 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.161 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:48.161 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125267+0000", 2024-04-25T20:10:48.161 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0038939230000000001 2024-04-25T20:10:48.161 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.161 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.161 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:48.161 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125309+0000", 2024-04-25T20:10:48.161 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1690999999999998e-05 2024-04-25T20:10:48.161 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.162 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.162 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:48.162 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125385+0000", 2024-04-25T20:10:48.162 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.6162999999999996e-05 2024-04-25T20:10:48.162 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.162 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.162 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:48.162 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126798+0000", 2024-04-25T20:10:48.162 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0014124680000000001 2024-04-25T20:10:48.162 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.162 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.162 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:48.162 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132264+0000", 2024-04-25T20:10:48.162 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0054661689999999999 2024-04-25T20:10:48.163 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.163 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.163 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:48.163 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132299+0000", 2024-04-25T20:10:48.163 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5272999999999999e-05 2024-04-25T20:10:48.163 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.163 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.163 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:48.163 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132329+0000", 2024-04-25T20:10:48.163 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.0080999999999999e-05 2024-04-25T20:10:48.163 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.163 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:48.163 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.164 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.164 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.164 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:48.164 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:10:48.164 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 89.969513445999993, 2024-04-25T20:10:48.164 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011015853000000001, 2024-04-25T20:10:48.164 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:48.164 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.164 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:48.164 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:48.164 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:48.164 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 139 2024-04-25T20:10:48.164 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.164 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:48.165 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.165 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:48.165 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:10:48.165 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.165 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.165 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.165 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:48.165 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121381+0000", 2024-04-25T20:10:48.165 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.165 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.165 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.165 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:48.165 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121380+0000", 2024-04-25T20:10:48.165 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:48.165 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.166 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.166 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:48.166 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121385+0000", 2024-04-25T20:10:48.166 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.092e-06 2024-04-25T20:10:48.166 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.166 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.166 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:48.166 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121390+0000", 2024-04-25T20:10:48.166 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.5990000000000001e-06 2024-04-25T20:10:48.166 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.166 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.166 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.166 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121397+0000", 2024-04-25T20:10:48.167 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.6279999999999999e-06 2024-04-25T20:10:48.167 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.167 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.167 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:48.167 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125737+0000", 2024-04-25T20:10:48.167 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0043406010000000004 2024-04-25T20:10:48.167 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.167 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.167 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:48.167 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125781+0000", 2024-04-25T20:10:48.167 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.3958999999999997e-05 2024-04-25T20:10:48.167 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.167 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.167 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:48.167 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.125849+0000", 2024-04-25T20:10:48.168 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8071999999999995e-05 2024-04-25T20:10:48.168 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.168 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.168 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:48.168 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.130908+0000", 2024-04-25T20:10:48.168 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0050586509999999999 2024-04-25T20:10:48.168 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.168 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.168 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:48.168 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132334+0000", 2024-04-25T20:10:48.168 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00142621 2024-04-25T20:10:48.168 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.168 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.168 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:48.169 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132369+0000", 2024-04-25T20:10:48.169 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4452000000000002e-05 2024-04-25T20:10:48.169 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.169 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.169 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:48.169 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132397+0000", 2024-04-25T20:10:48.169 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8065e-05 2024-04-25T20:10:48.169 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.169 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:48.169 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.169 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.169 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.169 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:48.169 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:48.170 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 89.969492402, 2024-04-25T20:10:48.170 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011069443, 2024-04-25T20:10:48.170 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:48.170 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.170 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:48.170 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:48.170 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:48.170 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 140 2024-04-25T20:10:48.170 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.170 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:48.170 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.170 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:48.170 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:48.170 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.171 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.171 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.171 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:48.171 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:48.171 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.171 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.171 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.171 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:48.171 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121402+0000", 2024-04-25T20:10:48.171 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:48.171 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.171 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.171 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:48.171 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121456+0000", 2024-04-25T20:10:48.171 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.4469000000000003e-05 2024-04-25T20:10:48.171 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.172 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.172 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:48.172 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121462+0000", 2024-04-25T20:10:48.172 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.1890000000000004e-06 2024-04-25T20:10:48.172 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.172 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.172 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.172 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121466+0000", 2024-04-25T20:10:48.172 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3909999999999998e-06 2024-04-25T20:10:48.172 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.172 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.172 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:48.172 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126071+0000", 2024-04-25T20:10:48.172 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0046048979999999996 2024-04-25T20:10:48.172 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.173 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.173 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:48.173 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126141+0000", 2024-04-25T20:10:48.173 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0621e-05 2024-04-25T20:10:48.173 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.173 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.173 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:48.173 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126260+0000", 2024-04-25T20:10:48.173 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000118621 2024-04-25T20:10:48.173 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.173 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.173 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:48.173 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.131270+0000", 2024-04-25T20:10:48.173 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0050096949999999998 2024-04-25T20:10:48.173 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.174 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.174 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:48.174 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132403+0000", 2024-04-25T20:10:48.174 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0011332779999999999 2024-04-25T20:10:48.174 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.174 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.174 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:48.174 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132443+0000", 2024-04-25T20:10:48.174 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0175000000000001e-05 2024-04-25T20:10:48.174 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.174 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.174 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:48.174 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132471+0000", 2024-04-25T20:10:48.174 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.8399e-05 2024-04-25T20:10:48.175 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.175 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:48.175 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.175 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.175 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.175 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:48.175 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:10:48.175 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 89.969423617000004, 2024-04-25T20:10:48.175 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.011071002, 2024-04-25T20:10:48.175 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:48.175 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.175 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:48.176 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:48.176 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:48.176 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 141 2024-04-25T20:10:48.176 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.176 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:48.176 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.176 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:48.176 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:10:48.176 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.176 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.176 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.176 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:48.176 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121471+0000", 2024-04-25T20:10:48.177 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.177 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.177 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.177 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:48.177 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121470+0000", 2024-04-25T20:10:48.177 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:48.177 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.177 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.177 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:48.177 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121473+0000", 2024-04-25T20:10:48.177 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.01e-06 2024-04-25T20:10:48.177 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.177 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.177 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:48.178 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121478+0000", 2024-04-25T20:10:48.178 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.515e-06 2024-04-25T20:10:48.178 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.178 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.178 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.178 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.121480+0000", 2024-04-25T20:10:48.178 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.5749999999999999e-06 2024-04-25T20:10:48.178 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.178 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.178 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:48.178 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126813+0000", 2024-04-25T20:10:48.178 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0053320399999999997 2024-04-25T20:10:48.178 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.179 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.179 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:48.179 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126854+0000", 2024-04-25T20:10:48.179 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.1582999999999998e-05 2024-04-25T20:10:48.179 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.179 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.179 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:48.179 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.126923+0000", 2024-04-25T20:10:48.179 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8544e-05 2024-04-25T20:10:48.179 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.179 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.179 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:48.179 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.131573+0000", 2024-04-25T20:10:48.180 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0046504659999999998 2024-04-25T20:10:48.180 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.180 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.180 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:48.180 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132478+0000", 2024-04-25T20:10:48.180 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00090442400000000003 2024-04-25T20:10:48.180 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.180 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.180 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:48.180 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132513+0000", 2024-04-25T20:10:48.180 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.5204999999999999e-05 2024-04-25T20:10:48.180 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.180 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.181 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:48.181 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.132542+0000", 2024-04-25T20:10:48.181 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.904e-05 2024-04-25T20:10:48.181 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.181 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:48.181 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.181 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.181 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.181 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:156 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:48.181 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:48.181 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 89.471599322000003, 2024-04-25T20:10:48.181 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090072280000000008, 2024-04-25T20:10:48.181 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:48.181 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.182 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:48.182 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:48.182 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:48.182 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 156 2024-04-25T20:10:48.182 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.182 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:48.182 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.182 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:48.182 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:48.182 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.182 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.182 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.182 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:48.183 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:48.183 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.183 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.183 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.183 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:48.183 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619295+0000", 2024-04-25T20:10:48.183 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:48.183 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.183 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.183 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:48.183 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619297+0000", 2024-04-25T20:10:48.183 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2979999999999999e-06 2024-04-25T20:10:48.183 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.183 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.184 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:48.184 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619300+0000", 2024-04-25T20:10:48.184 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.371e-06 2024-04-25T20:10:48.184 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.184 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.184 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.184 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619304+0000", 2024-04-25T20:10:48.184 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.304e-06 2024-04-25T20:10:48.184 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.184 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.184 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:48.184 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.622910+0000", 2024-04-25T20:10:48.184 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0036065049999999999 2024-04-25T20:10:48.184 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.184 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.185 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:48.185 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.622990+0000", 2024-04-25T20:10:48.185 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.9722999999999995e-05 2024-04-25T20:10:48.185 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.185 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.185 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:48.185 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623128+0000", 2024-04-25T20:10:48.185 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00013818399999999999 2024-04-25T20:10:48.185 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.185 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.185 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:48.185 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625155+0000", 2024-04-25T20:10:48.185 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0020267979999999998 2024-04-25T20:10:48.185 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.185 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.186 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:48.186 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628225+0000", 2024-04-25T20:10:48.186 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0030705680000000001 2024-04-25T20:10:48.186 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.186 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.186 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:48.186 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628268+0000", 2024-04-25T20:10:48.186 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.2404000000000003e-05 2024-04-25T20:10:48.186 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.186 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.186 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:48.186 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628302+0000", 2024-04-25T20:10:48.186 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4439e-05 2024-04-25T20:10:48.187 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.187 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:48.187 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.187 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.187 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.187 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:157 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:48.187 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:48.187 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 89.471587223, 2024-04-25T20:10:48.187 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090534889999999996, 2024-04-25T20:10:48.187 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:48.187 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.187 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:48.187 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:48.187 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:48.187 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 157 2024-04-25T20:10:48.188 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.188 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:48.188 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.188 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:48.188 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:48.188 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.188 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.188 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.188 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:48.188 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:48.188 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.188 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.188 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.188 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:48.188 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619307+0000", 2024-04-25T20:10:48.189 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:48.189 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.189 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.189 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:48.189 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619309+0000", 2024-04-25T20:10:48.189 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.841e-06 2024-04-25T20:10:48.189 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.189 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.189 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:48.189 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619312+0000", 2024-04-25T20:10:48.189 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.084e-06 2024-04-25T20:10:48.189 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.189 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.189 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.190 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619314+0000", 2024-04-25T20:10:48.190 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.3860000000000001e-06 2024-04-25T20:10:48.190 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.190 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.190 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:48.190 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623718+0000", 2024-04-25T20:10:48.190 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0044038050000000002 2024-04-25T20:10:48.190 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.190 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.190 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:48.190 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623791+0000", 2024-04-25T20:10:48.190 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3181000000000003e-05 2024-04-25T20:10:48.190 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.190 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.191 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:48.191 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.623922+0000", 2024-04-25T20:10:48.191 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00013117000000000001 2024-04-25T20:10:48.191 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.191 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.191 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:48.191 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625187+0000", 2024-04-25T20:10:48.191 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001264961 2024-04-25T20:10:48.191 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.191 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.191 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:48.191 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628308+0000", 2024-04-25T20:10:48.191 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.003120739 2024-04-25T20:10:48.192 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.192 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.192 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:48.192 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628331+0000", 2024-04-25T20:10:48.192 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2802999999999998e-05 2024-04-25T20:10:48.192 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.192 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.192 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:48.192 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628361+0000", 2024-04-25T20:10:48.192 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.9760999999999998e-05 2024-04-25T20:10:48.192 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.192 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:48.192 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.192 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.193 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.193 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:158 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:48.193 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:10:48.193 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 89.471576776000006, 2024-04-25T20:10:48.193 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0091076409999999997, 2024-04-25T20:10:48.193 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:48.193 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.193 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:48.193 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:48.193 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:48.193 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 158 2024-04-25T20:10:48.193 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.193 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:48.193 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.194 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:48.194 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:10:48.194 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.194 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.194 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.194 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:48.194 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619318+0000", 2024-04-25T20:10:48.194 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.194 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.194 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.194 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:48.194 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619317+0000", 2024-04-25T20:10:48.194 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967296 2024-04-25T20:10:48.194 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.195 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.195 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:48.195 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619364+0000", 2024-04-25T20:10:48.195 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.6572000000000001e-05 2024-04-25T20:10:48.195 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.195 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.195 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:48.195 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619371+0000", 2024-04-25T20:10:48.195 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3860000000000001e-06 2024-04-25T20:10:48.195 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.195 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.195 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.195 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619382+0000", 2024-04-25T20:10:48.195 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0173999999999999e-05 2024-04-25T20:10:48.195 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.196 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.196 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:48.196 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624216+0000", 2024-04-25T20:10:48.196 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048340350000000004 2024-04-25T20:10:48.196 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.196 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.196 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:48.196 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624260+0000", 2024-04-25T20:10:48.196 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.4558999999999998e-05 2024-04-25T20:10:48.196 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.196 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.196 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:48.196 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624333+0000", 2024-04-25T20:10:48.196 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2744999999999996e-05 2024-04-25T20:10:48.196 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.197 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.197 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:48.197 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.625465+0000", 2024-04-25T20:10:48.197 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001132454 2024-04-25T20:10:48.197 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.197 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.197 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:48.197 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628365+0000", 2024-04-25T20:10:48.197 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0028994950000000002 2024-04-25T20:10:48.197 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.197 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.197 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:48.197 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628386+0000", 2024-04-25T20:10:48.197 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.1367000000000001e-05 2024-04-25T20:10:48.197 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.198 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.198 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:48.198 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628425+0000", 2024-04-25T20:10:48.198 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.9087e-05 2024-04-25T20:10:48.198 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.198 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:48.198 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.198 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.198 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.198 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.4175.0:159 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-25T20:10:48.198 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:48.198 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 89.471508123000007, 2024-04-25T20:10:48.198 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0090948999999999995, 2024-04-25T20:10:48.199 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:48.199 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.199 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:48.199 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4175", 2024-04-25T20:10:48.199 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3365938615", 2024-04-25T20:10:48.199 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 159 2024-04-25T20:10:48.199 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.199 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:48.199 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.199 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:48.199 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:48.199 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.199 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.199 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.199 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:48.200 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:48.200 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:48.200 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.200 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.200 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:48.200 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619386+0000", 2024-04-25T20:10:48.200 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:48.200 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.200 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.200 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:48.200 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619389+0000", 2024-04-25T20:10:48.200 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.734e-06 2024-04-25T20:10:48.200 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.200 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.200 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:48.200 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619392+0000", 2024-04-25T20:10:48.201 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6219999999999998e-06 2024-04-25T20:10:48.201 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.201 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.201 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.201 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.619403+0000", 2024-04-25T20:10:48.201 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0987e-05 2024-04-25T20:10:48.201 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.201 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.201 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:48.201 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624667+0000", 2024-04-25T20:10:48.201 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0052633970000000004 2024-04-25T20:10:48.201 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.201 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.201 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:48.201 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624741+0000", 2024-04-25T20:10:48.202 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.4583000000000001e-05 2024-04-25T20:10:48.202 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.202 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.202 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 1", 2024-04-25T20:10:48.202 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.624857+0000", 2024-04-25T20:10:48.202 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000116118 2024-04-25T20:10:48.202 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.202 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.202 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:48.202 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.626013+0000", 2024-04-25T20:10:48.202 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.001155473 2024-04-25T20:10:48.202 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.202 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.202 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:48.202 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628432+0000", 2024-04-25T20:10:48.203 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0024189379999999998 2024-04-25T20:10:48.203 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.203 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.203 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:48.203 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628454+0000", 2024-04-25T20:10:48.203 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.2059999999999999e-05 2024-04-25T20:10:48.203 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:48.203 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:48.203 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:48.203 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:09:18.628481+0000", 2024-04-25T20:10:48.203 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.7300999999999998e-05 2024-04-25T20:10:48.203 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.203 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:48.204 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.204 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:48.204 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:48.204 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:48.205 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:48.205+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:48.205 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T20:10:48.306 INFO:tasks.workunit.client.0.smithi039.stderr:+ journal1=145ee3bcd187 2024-04-25T20:10:48.306 INFO:tasks.workunit.client.0.smithi039.stderr:+ save_commit_position 145ee3bcd187 2024-04-25T20:10:48.307 INFO:tasks.workunit.client.0.smithi039.stderr:+ local journal=145ee3bcd187 2024-04-25T20:10:48.307 INFO:tasks.workunit.client.0.smithi039.stderr:+ rados -p rbd getomapval journal.145ee3bcd187 client_ /tmp/rbd_journal47106/145ee3bcd187.client_.omap 2024-04-25T20:10:48.307 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:48.305+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:48.312 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:48.312 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:48.312 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:48.312 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:48.321 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T20:10:48.342 INFO:tasks.workunit.client.0.smithi039.stderr:Writing to /tmp/rbd_journal47106/145ee3bcd187.client_.omap 2024-04-25T20:10:48.345 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd journal import --dest testrbdjournal471061 /tmp/rbd_journal47106/journal.export 2024-04-25T20:10:48.399 INFO:tasks.workunit.client.0.smithi039.stdout:12 entries processed, 0 errors 2024-04-25T20:10:48.399 INFO:tasks.workunit.client.0.smithi039.stdout:Waiting for journal append to complete... 2024-04-25T20:10:48.407 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:48.406+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:48.409 INFO:tasks.workunit.client.0.smithi039.stderr:2024-04-25T20:10:48.409+0000 7fc8027ff640 -1 JournalMetadata: 0x558b32754a80 failed to locate client: IMPORT 2024-04-25T20:10:48.412 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd snap create testrbdjournal471061@test 2024-04-25T20:10:48.433 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:48.433 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:48.433 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:48.433 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:48.433 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:48.443 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T20:10:48.509 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:48.508+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:48.574 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:48.574 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:48.574 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-04-25T20:10:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4928.0:1 1.6 e59/14 1:67e51d9e:::journal.134bdebc9a96:head v 59'15, mlcod=59'13)", 2024-04-25T20:10:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:47.780044+0000", 2024-04-25T20:10:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.79291066300000002, 2024-04-25T20:10:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00088104299999999997, 2024-04-25T20:10:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.780044+0000", 2024-04-25T20:10:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.575 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.780044+0000", 2024-04-25T20:10:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.780043+0000", 2024-04-25T20:10:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999986 2024-04-25T20:10:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.780055+0000", 2024-04-25T20:10:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2918000000000001e-05 2024-04-25T20:10:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:48.576 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.780073+0000", 2024-04-25T20:10:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7394999999999999e-05 2024-04-25T20:10:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.780083+0000", 2024-04-25T20:10:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0611999999999999e-05 2024-04-25T20:10:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.780130+0000", 2024-04-25T20:10:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.6489999999999997e-05 2024-04-25T20:10:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:48.577 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.780149+0000", 2024-04-25T20:10:48.578 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8691000000000001e-05 2024-04-25T20:10:48.578 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.578 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.578 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:48.578 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.780887+0000", 2024-04-25T20:10:48.578 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00073860500000000003 2024-04-25T20:10:48.578 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.578 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.578 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:48.578 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.780925+0000", 2024-04-25T20:10:48.578 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7880000000000003e-05 2024-04-25T20:10:48.578 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.578 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:48.578 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.579 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.579 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.579 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.5196.0:1 1.6 e59/14 1:67e51d9e:::journal.134bdebc9a96:head v 59'16, mlcod=59'14)", 2024-04-25T20:10:48.579 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:47.869275+0000", 2024-04-25T20:10:48.579 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.70367932399999999, 2024-04-25T20:10:48.579 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00066712800000000001, 2024-04-25T20:10:48.579 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:48.579 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.579 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:48.579 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.579 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:48.579 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.869275+0000", 2024-04-25T20:10:48.579 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.579 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.869275+0000", 2024-04-25T20:10:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.869274+0000", 2024-04-25T20:10:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999981 2024-04-25T20:10:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.869288+0000", 2024-04-25T20:10:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4511999999999999e-05 2024-04-25T20:10:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.869308+0000", 2024-04-25T20:10:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9956999999999999e-05 2024-04-25T20:10:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.869319+0000", 2024-04-25T20:10:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0915e-05 2024-04-25T20:10:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.869364+0000", 2024-04-25T20:10:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5206000000000001e-05 2024-04-25T20:10:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.869384+0000", 2024-04-25T20:10:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9470000000000002e-05 2024-04-25T20:10:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.869916+0000", 2024-04-25T20:10:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00053236799999999997 2024-04-25T20:10:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.869943+0000", 2024-04-25T20:10:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6537999999999999e-05 2024-04-25T20:10:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.5196.0:3 1.6 e59/14 1:67e51d9e:::journal.134bdebc9a96:head v 59'17, mlcod=59'15)", 2024-04-25T20:10:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:47.870706+0000", 2024-04-25T20:10:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.70224858700000004, 2024-04-25T20:10:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00050497400000000005, 2024-04-25T20:10:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.870706+0000", 2024-04-25T20:10:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.870706+0000", 2024-04-25T20:10:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.870705+0000", 2024-04-25T20:10:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.870712+0000", 2024-04-25T20:10:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3969999999999999e-06 2024-04-25T20:10:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.870724+0000", 2024-04-25T20:10:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.218e-05 2024-04-25T20:10:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.870731+0000", 2024-04-25T20:10:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9399999999999996e-06 2024-04-25T20:10:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:48.585 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.870757+0000", 2024-04-25T20:10:48.586 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5877e-05 2024-04-25T20:10:48.586 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.586 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.586 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:48.586 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.870768+0000", 2024-04-25T20:10:48.586 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1669e-05 2024-04-25T20:10:48.586 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.586 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.586 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:48.586 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.871189+0000", 2024-04-25T20:10:48.586 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000420491 2024-04-25T20:10:48.586 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.586 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.586 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.871211+0000", 2024-04-25T20:10:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2192999999999999e-05 2024-04-25T20:10:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.5196.0:12 1.6 e59/14 1:67e51d9e:::journal.134bdebc9a96:head v 59'18, mlcod=59'16)", 2024-04-25T20:10:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:47.876158+0000", 2024-04-25T20:10:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.69679721400000005, 2024-04-25T20:10:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00051606099999999995, 2024-04-25T20:10:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.587 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.876158+0000", 2024-04-25T20:10:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.876158+0000", 2024-04-25T20:10:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.876157+0000", 2024-04-25T20:10:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:48.588 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.876170+0000", 2024-04-25T20:10:48.589 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3061000000000001e-05 2024-04-25T20:10:48.589 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.589 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.589 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:48.589 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.876180+0000", 2024-04-25T20:10:48.589 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.9399999999999997e-06 2024-04-25T20:10:48.589 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.589 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.589 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.589 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.876184+0000", 2024-04-25T20:10:48.589 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3440000000000003e-06 2024-04-25T20:10:48.589 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.589 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.589 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:48.589 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.876211+0000", 2024-04-25T20:10:48.590 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6458999999999999e-05 2024-04-25T20:10:48.590 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.590 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.590 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:48.590 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.876223+0000", 2024-04-25T20:10:48.590 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.185e-05 2024-04-25T20:10:48.590 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.590 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.590 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:48.590 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.876647+0000", 2024-04-25T20:10:48.590 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00042474500000000003 2024-04-25T20:10:48.590 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.590 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.590 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:48.590 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.876674+0000", 2024-04-25T20:10:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6225999999999999e-05 2024-04-25T20:10:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.5196.0:16 1.6 e59/14 1:67e51d9e:::journal.134bdebc9a96:head v 59'19, mlcod=59'17)", 2024-04-25T20:10:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:47.880217+0000", 2024-04-25T20:10:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.69273755400000003, 2024-04-25T20:10:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00065820899999999999, 2024-04-25T20:10:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:48.591 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.880217+0000", 2024-04-25T20:10:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.880217+0000", 2024-04-25T20:10:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.880216+0000", 2024-04-25T20:10:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999986 2024-04-25T20:10:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.880227+0000", 2024-04-25T20:10:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1813e-05 2024-04-25T20:10:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.880262+0000", 2024-04-25T20:10:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4595999999999999e-05 2024-04-25T20:10:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.880266+0000", 2024-04-25T20:10:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.313e-06 2024-04-25T20:10:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.880294+0000", 2024-04-25T20:10:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7248999999999999e-05 2024-04-25T20:10:48.593 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.594 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.594 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:48.594 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.880307+0000", 2024-04-25T20:10:48.594 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3042e-05 2024-04-25T20:10:48.594 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.594 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.594 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:48.594 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.880855+0000", 2024-04-25T20:10:48.594 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00054833799999999999 2024-04-25T20:10:48.594 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.594 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.594 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:48.594 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.880875+0000", 2024-04-25T20:10:48.594 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0367999999999999e-05 2024-04-25T20:10:48.594 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4946.0:1 1.6 e59/14 1:67e51d9e:::journal.134bdebc9a96:head v 59'20, mlcod=59'18)", 2024-04-25T20:10:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:47.918501+0000", 2024-04-25T20:10:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.65445397500000002, 2024-04-25T20:10:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000612763, 2024-04-25T20:10:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.918501+0000", 2024-04-25T20:10:48.595 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.596 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.596 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.596 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:48.596 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.918501+0000", 2024-04-25T20:10:48.596 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.596 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.596 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.596 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:48.596 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.918499+0000", 2024-04-25T20:10:48.596 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999986 2024-04-25T20:10:48.596 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.596 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.596 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:48.596 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.918510+0000", 2024-04-25T20:10:48.597 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0156000000000001e-05 2024-04-25T20:10:48.597 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.597 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.597 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:48.597 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.918521+0000", 2024-04-25T20:10:48.597 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.116e-05 2024-04-25T20:10:48.597 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.597 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.597 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.597 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.918528+0000", 2024-04-25T20:10:48.597 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0870000000000001e-06 2024-04-25T20:10:48.597 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.597 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.597 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:48.597 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.918573+0000", 2024-04-25T20:10:48.598 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4712999999999997e-05 2024-04-25T20:10:48.598 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.598 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.598 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:48.598 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.918590+0000", 2024-04-25T20:10:48.598 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7346999999999998e-05 2024-04-25T20:10:48.598 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.598 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.598 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:48.598 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.919089+0000", 2024-04-25T20:10:48.598 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000499345 2024-04-25T20:10:48.598 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.598 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.598 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.919114+0000", 2024-04-25T20:10:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4239e-05 2024-04-25T20:10:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4946.0:16 1.6 e59/14 1:67e51d9e:::journal.134bdebc9a96:head v 59'23, mlcod=59'21)", 2024-04-25T20:10:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:47.925570+0000", 2024-04-25T20:10:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.647384984, 2024-04-25T20:10:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00057289299999999999, 2024-04-25T20:10:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:48.599 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.925570+0000", 2024-04-25T20:10:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.925570+0000", 2024-04-25T20:10:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.925569+0000", 2024-04-25T20:10:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.600 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.925584+0000", 2024-04-25T20:10:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4439e-05 2024-04-25T20:10:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.925593+0000", 2024-04-25T20:10:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.3899999999999999e-06 2024-04-25T20:10:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.925597+0000", 2024-04-25T20:10:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4000000000000002e-06 2024-04-25T20:10:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.601 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.602 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:48.602 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.925622+0000", 2024-04-25T20:10:48.602 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4357000000000001e-05 2024-04-25T20:10:48.602 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.602 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.602 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:48.602 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.925632+0000", 2024-04-25T20:10:48.602 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0363000000000001e-05 2024-04-25T20:10:48.602 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.602 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.602 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:48.602 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.926122+0000", 2024-04-25T20:10:48.602 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00048937099999999995 2024-04-25T20:10:48.602 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:47.926143+0000", 2024-04-25T20:10:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1098e-05 2024-04-25T20:10:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.5214.0:7 1.0 e59/14 1:07a5e98e:::rbd_header.145ee3bcd187:head v 59'1, mlcod=0'0)", 2024-04-25T20:10:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:48.046574+0000", 2024-04-25T20:10:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.52638042100000004, 2024-04-25T20:10:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00057989000000000001, 2024-04-25T20:10:48.603 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.046574+0000", 2024-04-25T20:10:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.046574+0000", 2024-04-25T20:10:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.046573+0000", 2024-04-25T20:10:48.604 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.046580+0000", 2024-04-25T20:10:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.0339999999999999e-06 2024-04-25T20:10:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.046591+0000", 2024-04-25T20:10:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0302999999999999e-05 2024-04-25T20:10:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.046595+0000", 2024-04-25T20:10:48.605 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5410000000000002e-06 2024-04-25T20:10:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.046631+0000", 2024-04-25T20:10:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.5945000000000002e-05 2024-04-25T20:10:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.046648+0000", 2024-04-25T20:10:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6402999999999999e-05 2024-04-25T20:10:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.047119+0000", 2024-04-25T20:10:48.606 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00047184899999999999 2024-04-25T20:10:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.047154+0000", 2024-04-25T20:10:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4761000000000001e-05 2024-04-25T20:10:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.5214.0:9 1.0 e59/14 1:172f2fe1:::journal.145ee3bcd187:head v 59'2, mlcod=0'0)", 2024-04-25T20:10:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:48.048918+0000", 2024-04-25T20:10:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.52403645099999996, 2024-04-25T20:10:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00085295200000000003, 2024-04-25T20:10:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.607 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.048918+0000", 2024-04-25T20:10:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.048918+0000", 2024-04-25T20:10:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.048915+0000", 2024-04-25T20:10:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999971 2024-04-25T20:10:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.048923+0000", 2024-04-25T20:10:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7870000000000002e-06 2024-04-25T20:10:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.048931+0000", 2024-04-25T20:10:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1219999999999995e-06 2024-04-25T20:10:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.048937+0000", 2024-04-25T20:10:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8560000000000003e-06 2024-04-25T20:10:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.609 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.048977+0000", 2024-04-25T20:10:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9823999999999997e-05 2024-04-25T20:10:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.048999+0000", 2024-04-25T20:10:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2487999999999999e-05 2024-04-25T20:10:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.049738+0000", 2024-04-25T20:10:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000738823 2024-04-25T20:10:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.610 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.049771+0000", 2024-04-25T20:10:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2932000000000003e-05 2024-04-25T20:10:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4982.0:1 1.0 e59/14 1:172f2fe1:::journal.145ee3bcd187:head v 59'5, mlcod=59'3)", 2024-04-25T20:10:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:48.395241+0000", 2024-04-25T20:10:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.17771374300000001, 2024-04-25T20:10:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00082266999999999996, 2024-04-25T20:10:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.611 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.395241+0000", 2024-04-25T20:10:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.395241+0000", 2024-04-25T20:10:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.395239+0000", 2024-04-25T20:10:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999986 2024-04-25T20:10:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.612 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.613 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:48.613 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.395253+0000", 2024-04-25T20:10:48.613 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3072e-05 2024-04-25T20:10:48.613 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.613 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.613 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:48.613 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.395271+0000", 2024-04-25T20:10:48.613 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8907999999999999e-05 2024-04-25T20:10:48.613 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.613 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.613 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.613 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.395283+0000", 2024-04-25T20:10:48.613 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1517e-05 2024-04-25T20:10:48.613 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.614 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.614 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:48.614 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.395327+0000", 2024-04-25T20:10:48.614 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3939000000000001e-05 2024-04-25T20:10:48.614 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.614 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.614 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:48.614 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.395347+0000", 2024-04-25T20:10:48.614 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.9848999999999999e-05 2024-04-25T20:10:48.614 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.614 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.614 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.396027+0000", 2024-04-25T20:10:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00068058699999999997 2024-04-25T20:10:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.396064+0000", 2024-04-25T20:10:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6344000000000003e-05 2024-04-25T20:10:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.615 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4982.0:3 1.0 e59/14 1:172f2fe1:::journal.145ee3bcd187:head v 59'6, mlcod=59'4)", 2024-04-25T20:10:48.616 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:48.396947+0000", 2024-04-25T20:10:48.616 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.176007894, 2024-04-25T20:10:48.616 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00066376100000000002, 2024-04-25T20:10:48.616 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:48.616 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.616 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:48.616 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.616 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:48.616 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.396947+0000", 2024-04-25T20:10:48.616 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.616 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.616 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.616 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:48.616 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.396947+0000", 2024-04-25T20:10:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.396946+0000", 2024-04-25T20:10:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.396952+0000", 2024-04-25T20:10:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4010000000000002e-06 2024-04-25T20:10:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.617 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.396962+0000", 2024-04-25T20:10:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.7040000000000006e-06 2024-04-25T20:10:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.396968+0000", 2024-04-25T20:10:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.3959999999999997e-06 2024-04-25T20:10:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.396993+0000", 2024-04-25T20:10:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5922999999999999e-05 2024-04-25T20:10:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.618 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.397006+0000", 2024-04-25T20:10:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2792999999999999e-05 2024-04-25T20:10:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.397582+0000", 2024-04-25T20:10:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00057583299999999999 2024-04-25T20:10:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.397611+0000", 2024-04-25T20:10:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8515999999999999e-05 2024-04-25T20:10:48.619 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.620 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:48.620 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.620 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.620 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.620 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4982.0:8 1.2 1:54a62035:::journal_data.1.145ee3bcd187.1:head [call journal.append in=13b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-25T20:10:48.620 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:48.399422+0000", 2024-04-25T20:10:48.620 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.17353244000000001, 2024-04-25T20:10:48.620 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0066612119999999997, 2024-04-25T20:10:48.620 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:48.620 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.620 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-04-25T20:10:48.620 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4982", 2024-04-25T20:10:48.620 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1605752383", 2024-04-25T20:10:48.620 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 8 2024-04-25T20:10:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.399422+0000", 2024-04-25T20:10:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.399422+0000", 2024-04-25T20:10:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.399421+0000", 2024-04-25T20:10:48.621 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999981 2024-04-25T20:10:48.622 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.622 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.622 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:48.622 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.399433+0000", 2024-04-25T20:10:48.622 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2313000000000001e-05 2024-04-25T20:10:48.622 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.622 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.622 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:48.622 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.399446+0000", 2024-04-25T20:10:48.622 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.275e-05 2024-04-25T20:10:48.622 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.622 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.622 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.399454+0000", 2024-04-25T20:10:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9310000000000008e-06 2024-04-25T20:10:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.399510+0000", 2024-04-25T20:10:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.6014999999999998e-05 2024-04-25T20:10:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.399642+0000", 2024-04-25T20:10:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000132223 2024-04-25T20:10:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-04-25T20:10:48.624 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.399807+0000", 2024-04-25T20:10:48.624 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000164986 2024-04-25T20:10:48.624 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.624 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.624 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:48.624 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.405964+0000", 2024-04-25T20:10:48.624 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0061574380000000003 2024-04-25T20:10:48.624 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.624 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.624 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-04-25T20:10:48.624 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.405976+0000", 2024-04-25T20:10:48.624 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1345000000000001e-05 2024-04-25T20:10:48.624 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.624 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.406035+0000", 2024-04-25T20:10:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9290999999999999e-05 2024-04-25T20:10:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.406084+0000", 2024-04-25T20:10:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8619999999999999e-05 2024-04-25T20:10:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4982.0:12 1.2 1:54a62035:::journal_data.1.145ee3bcd187.1:head [call journal.append in=13b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-25T20:10:48.625 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:48.399477+0000", 2024-04-25T20:10:48.626 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.17347823500000001, 2024-04-25T20:10:48.626 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0067547240000000001, 2024-04-25T20:10:48.626 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:48.626 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.626 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-04-25T20:10:48.626 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4982", 2024-04-25T20:10:48.626 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1605752383", 2024-04-25T20:10:48.626 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 12 2024-04-25T20:10:48.626 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.626 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:48.626 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.626 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:48.626 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.399477+0000", 2024-04-25T20:10:48.626 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.399477+0000", 2024-04-25T20:10:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.399476+0000", 2024-04-25T20:10:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.399483+0000", 2024-04-25T20:10:48.627 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.9909999999999996e-06 2024-04-25T20:10:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.399490+0000", 2024-04-25T20:10:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.5190000000000003e-06 2024-04-25T20:10:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.399496+0000", 2024-04-25T20:10:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1610000000000002e-06 2024-04-25T20:10:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400138+0000", 2024-04-25T20:10:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00064211800000000001 2024-04-25T20:10:48.628 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.629 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.629 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:48.629 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400240+0000", 2024-04-25T20:10:48.629 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000101884 2024-04-25T20:10:48.629 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.629 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.629 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-04-25T20:10:48.629 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400447+0000", 2024-04-25T20:10:48.629 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000207708 2024-04-25T20:10:48.629 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.629 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.629 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-04-25T20:10:48.629 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.406098+0000", 2024-04-25T20:10:48.629 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.005651115 2024-04-25T20:10:48.629 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.630 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.630 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:48.630 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.406137+0000", 2024-04-25T20:10:48.630 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8454999999999997e-05 2024-04-25T20:10:48.630 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.630 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.630 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:48.630 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.406192+0000", 2024-04-25T20:10:48.630 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5284999999999997e-05 2024-04-25T20:10:48.630 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.630 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.630 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.406231+0000", 2024-04-25T20:10:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9001000000000001e-05 2024-04-25T20:10:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_op(client.4982.0:16 1.2 1:54a62035:::journal_data.1.145ee3bcd187.1:head [call journal.append in=13b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-04-25T20:10:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:48.399637+0000", 2024-04-25T20:10:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.17331734100000001, 2024-04-25T20:10:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0076127429999999999, 2024-04-25T20:10:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: "client_info": { 2024-04-25T20:10:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: "client": "client.4982", 2024-04-25T20:10:48.631 INFO:teuthology.orchestra.run.smithi039.stdout: "client_addr": "172.21.15.39:0/1605752383", 2024-04-25T20:10:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: "tid": 16 2024-04-25T20:10:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.399637+0000", 2024-04-25T20:10:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.399637+0000", 2024-04-25T20:10:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:48.632 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.399637+0000", 2024-04-25T20:10:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.399645+0000", 2024-04-25T20:10:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.858e-06 2024-04-25T20:10:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.399655+0000", 2024-04-25T20:10:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0611999999999999e-05 2024-04-25T20:10:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.399662+0000", 2024-04-25T20:10:48.633 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.331e-06 2024-04-25T20:10:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.405251+0000", 2024-04-25T20:10:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0055898019999999996 2024-04-25T20:10:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.405357+0000", 2024-04-25T20:10:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000105939 2024-04-25T20:10:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "waiting for subops from 4", 2024-04-25T20:10:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.405542+0000", 2024-04-25T20:10:48.634 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000184871 2024-04-25T20:10:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "op_commit", 2024-04-25T20:10:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.406485+0000", 2024-04-25T20:10:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00094321300000000004 2024-04-25T20:10:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.407200+0000", 2024-04-25T20:10:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00071485199999999998 2024-04-25T20:10:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.407225+0000", 2024-04-25T20:10:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4913999999999999e-05 2024-04-25T20:10:48.635 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.636 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.636 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:48.636 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.407250+0000", 2024-04-25T20:10:48.636 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.497e-05 2024-04-25T20:10:48.636 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.636 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:48.636 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.636 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.636 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.636 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop_reply(client.4982.0:8 1.2 e59/51 ondisk, result = 0)", 2024-04-25T20:10:48.636 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:48.402247+0000", 2024-04-25T20:10:48.636 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.17070769899999999, 2024-04-25T20:10:48.636 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0038432240000000001, 2024-04-25T20:10:48.636 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:48.636 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-04-25T20:10:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.402247+0000", 2024-04-25T20:10:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.402247+0000", 2024-04-25T20:10:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.402246+0000", 2024-04-25T20:10:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:48.637 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.638 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.638 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:48.638 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.402251+0000", 2024-04-25T20:10:48.638 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1409999999999998e-06 2024-04-25T20:10:48.638 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.638 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.638 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:48.638 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.402260+0000", 2024-04-25T20:10:48.638 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6670000000000008e-06 2024-04-25T20:10:48.638 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.638 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.638 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.638 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.402266+0000", 2024-04-25T20:10:48.638 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.6749999999999999e-06 2024-04-25T20:10:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.405948+0000", 2024-04-25T20:10:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0036817080000000001 2024-04-25T20:10:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.405958+0000", 2024-04-25T20:10:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0001e-05 2024-04-25T20:10:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.406090+0000", 2024-04-25T20:10:48.639 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00013267199999999999 2024-04-25T20:10:48.640 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.640 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:48.640 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.640 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.640 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.640 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop_reply(client.4982.0:12 1.2 e59/51 ondisk, result = 0)", 2024-04-25T20:10:48.640 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:48.402272+0000", 2024-04-25T20:10:48.640 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.170682528, 2024-04-25T20:10:48.640 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0039671180000000004, 2024-04-25T20:10:48.640 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:48.640 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-04-25T20:10:48.640 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:48.640 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.402272+0000", 2024-04-25T20:10:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.402272+0000", 2024-04-25T20:10:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.402272+0000", 2024-04-25T20:10:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967296 2024-04-25T20:10:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.641 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.402274+0000", 2024-04-25T20:10:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5e-06 2024-04-25T20:10:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.402277+0000", 2024-04-25T20:10:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.394e-06 2024-04-25T20:10:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.402287+0000", 2024-04-25T20:10:48.642 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.8579999999999995e-06 2024-04-25T20:10:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.406124+0000", 2024-04-25T20:10:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0038368590000000002 2024-04-25T20:10:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.406131+0000", 2024-04-25T20:10:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.4320000000000002e-06 2024-04-25T20:10:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.406239+0000", 2024-04-25T20:10:48.643 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000108267 2024-04-25T20:10:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4982.0:23 1.0 e59/14 1:172f2fe1:::journal.145ee3bcd187:head v 59'8, mlcod=59'6)", 2024-04-25T20:10:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:48.409391+0000", 2024-04-25T20:10:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.16356358600000001, 2024-04-25T20:10:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00053815099999999995, 2024-04-25T20:10:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:48.644 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.409391+0000", 2024-04-25T20:10:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.409391+0000", 2024-04-25T20:10:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.409391+0000", 2024-04-25T20:10:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:48.645 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.409406+0000", 2024-04-25T20:10:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5228000000000001e-05 2024-04-25T20:10:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.409414+0000", 2024-04-25T20:10:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.4840000000000004e-06 2024-04-25T20:10:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.409419+0000", 2024-04-25T20:10:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4469999999999999e-06 2024-04-25T20:10:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:48.646 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.409449+0000", 2024-04-25T20:10:48.647 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0039000000000002e-05 2024-04-25T20:10:48.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.647 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.647 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:48.647 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.409462+0000", 2024-04-25T20:10:48.647 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2762000000000001e-05 2024-04-25T20:10:48.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.647 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.647 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:48.647 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.409897+0000", 2024-04-25T20:10:48.647 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000435103 2024-04-25T20:10:48.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.647 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.647 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:48.648 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.409929+0000", 2024-04-25T20:10:48.648 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2591e-05 2024-04-25T20:10:48.648 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.648 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:48.648 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.648 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.648 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.648 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.5256.0:12 1.0 e59/14 1:07a5e98e:::rbd_header.145ee3bcd187:head v 59'9, mlcod=59'7)", 2024-04-25T20:10:48.648 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:48.453662+0000", 2024-04-25T20:10:48.648 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.11929248000000001, 2024-04-25T20:10:48.648 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00061123099999999995, 2024-04-25T20:10:48.648 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:48.648 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.648 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:48.648 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.648 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:48.649 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.453662+0000", 2024-04-25T20:10:48.649 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.649 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.649 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.649 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:48.649 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.453662+0000", 2024-04-25T20:10:48.649 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.649 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.649 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.649 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:48.649 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.453662+0000", 2024-04-25T20:10:48.649 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:48.649 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.649 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.649 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:48.649 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.453667+0000", 2024-04-25T20:10:48.650 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8499999999999999e-06 2024-04-25T20:10:48.650 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.650 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.650 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:48.650 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.453677+0000", 2024-04-25T20:10:48.650 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.6330000000000008e-06 2024-04-25T20:10:48.650 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.650 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.650 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.650 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.453682+0000", 2024-04-25T20:10:48.650 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5789999999999997e-06 2024-04-25T20:10:48.650 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.650 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.650 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:48.651 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.453719+0000", 2024-04-25T20:10:48.651 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6995000000000001e-05 2024-04-25T20:10:48.651 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.651 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.651 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:48.651 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.453736+0000", 2024-04-25T20:10:48.651 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6915999999999999e-05 2024-04-25T20:10:48.651 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.651 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.651 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:48.651 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.454246+0000", 2024-04-25T20:10:48.651 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00051069400000000001 2024-04-25T20:10:48.651 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.651 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.651 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:48.651 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.454273+0000", 2024-04-25T20:10:48.652 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7175e-05 2024-04-25T20:10:48.652 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.652 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:48.652 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.652 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.652 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.652 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.5256.0:14 1.0 e59/14 1:07a5e98e:::rbd_header.145ee3bcd187:head v 59'10, mlcod=59'8)", 2024-04-25T20:10:48.652 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:48.455126+0000", 2024-04-25T20:10:48.652 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.11782920700000001, 2024-04-25T20:10:48.652 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00053371699999999996, 2024-04-25T20:10:48.652 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:48.652 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.652 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:48.652 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.652 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:48.652 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.455126+0000", 2024-04-25T20:10:48.652 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.653 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.653 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.653 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:48.653 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.455126+0000", 2024-04-25T20:10:48.653 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.653 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.653 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.653 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:48.653 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.455125+0000", 2024-04-25T20:10:48.653 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:48.653 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.653 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.653 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:48.653 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.455129+0000", 2024-04-25T20:10:48.653 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4399999999999998e-06 2024-04-25T20:10:48.653 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.654 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.654 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:48.654 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.455138+0000", 2024-04-25T20:10:48.654 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7960000000000001e-06 2024-04-25T20:10:48.654 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.654 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.654 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.654 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.455143+0000", 2024-04-25T20:10:48.654 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.3959999999999999e-06 2024-04-25T20:10:48.654 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.654 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.654 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:48.654 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.455171+0000", 2024-04-25T20:10:48.654 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7883000000000001e-05 2024-04-25T20:10:48.655 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.655 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.655 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:48.655 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.455182+0000", 2024-04-25T20:10:48.655 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1790000000000001e-05 2024-04-25T20:10:48.655 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.655 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.655 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:48.655 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.455639+0000", 2024-04-25T20:10:48.655 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00045651200000000001 2024-04-25T20:10:48.655 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.655 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.655 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:48.655 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.455659+0000", 2024-04-25T20:10:48.655 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0414000000000001e-05 2024-04-25T20:10:48.655 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.655 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:48.656 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.656 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.656 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.656 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.5256.0:41 1.0 e59/14 1:172f2fe1:::journal.145ee3bcd187:head v 59'13, mlcod=59'11)", 2024-04-25T20:10:48.656 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:48.466516+0000", 2024-04-25T20:10:48.656 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.10643836499999999, 2024-04-25T20:10:48.656 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00050917099999999999, 2024-04-25T20:10:48.656 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:48.656 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:48.656 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:48.656 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.656 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:48.656 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.466516+0000", 2024-04-25T20:10:48.656 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.656 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.657 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.657 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:48.657 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.466516+0000", 2024-04-25T20:10:48.657 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:48.657 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.657 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.657 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:48.657 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.466516+0000", 2024-04-25T20:10:48.657 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:48.657 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.657 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.657 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:48.657 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.466520+0000", 2024-04-25T20:10:48.657 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4850000000000003e-06 2024-04-25T20:10:48.657 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.657 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.658 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:48.658 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.466529+0000", 2024-04-25T20:10:48.658 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.2290000000000003e-06 2024-04-25T20:10:48.658 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.658 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.658 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:48.658 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.466534+0000", 2024-04-25T20:10:48.658 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.6869999999999999e-06 2024-04-25T20:10:48.658 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.658 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.658 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:48.658 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.466558+0000", 2024-04-25T20:10:48.658 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.338e-05 2024-04-25T20:10:48.658 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.659 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.659 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:48.659 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.466569+0000", 2024-04-25T20:10:48.659 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0898e-05 2024-04-25T20:10:48.659 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.659 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.659 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:48.659 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.467005+0000", 2024-04-25T20:10:48.659 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00043654700000000002 2024-04-25T20:10:48.659 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:48.659 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:48.659 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:48.659 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.467026+0000", 2024-04-25T20:10:48.659 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0414000000000001e-05 2024-04-25T20:10:48.659 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.660 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:48.660 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.660 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:48.660 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:48.660 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:48.661 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T20:10:48.662 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:48.661+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:48.762 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:48.762+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:48.767 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:48.767 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:48.767 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:48.767 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:48.776 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T20:10:48.864 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:48.863+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:48.923 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:48.924 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:48.924 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:48.924 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:48.924 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:48.932 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T20:10:48.965 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:48.965+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:48.993 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is unset 2024-04-25T20:10:49.003 INFO:tasks.workunit.client.0.smithi039.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2024-04-25T20:10:49.005 INFO:tasks.workunit.client.0.smithi039.stderr:2024-04-25T20:10:49.004+0000 7fb6d4bab640 0 -- 172.21.15.39:0/1417702530 >> [v2:172.21.15.135:6816/31737,v1:172.21.15.135:6817/31737] conn(0x7fb6b0008be0 msgr2=0x7fb6b80747f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-25T20:10:49.042 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:49.042 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:49.042 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:49.042 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-04-25T20:10:49.043 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.043 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:30 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'16, mlcod=16'14)", 2024-04-25T20:10:49.043 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:10:49.043 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 91.874172114999993, 2024-04-25T20:10:49.043 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0054087670000000001, 2024-04-25T20:10:49.043 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:49.043 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:49.043 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:49.043 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.043 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:49.043 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:10:49.043 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.043 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.043 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.044 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:49.044 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:10:49.044 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.044 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.044 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.044 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:49.044 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167142+0000", 2024-04-25T20:10:49.044 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999976 2024-04-25T20:10:49.044 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.044 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.044 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:49.044 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167162+0000", 2024-04-25T20:10:49.044 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0109e-05 2024-04-25T20:10:49.044 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.045 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.045 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:49.045 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167188+0000", 2024-04-25T20:10:49.045 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6146e-05 2024-04-25T20:10:49.045 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.045 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.045 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:49.045 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167199+0000", 2024-04-25T20:10:49.045 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0767e-05 2024-04-25T20:10:49.045 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.045 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.045 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:49.045 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167256+0000", 2024-04-25T20:10:49.045 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7392000000000002e-05 2024-04-25T20:10:49.045 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.046 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.046 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:49.046 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167283+0000", 2024-04-25T20:10:49.046 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6577e-05 2024-04-25T20:10:49.046 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.046 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.046 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:49.046 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.172510+0000", 2024-04-25T20:10:49.046 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0052276759999999997 2024-04-25T20:10:49.046 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.046 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.046 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:49.046 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.172553+0000", 2024-04-25T20:10:49.046 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2444999999999998e-05 2024-04-25T20:10:49.047 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.047 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:49.047 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.047 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.047 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.047 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:31 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-04-25T20:10:49.047 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:49.048 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 91.873556037, 2024-04-25T20:10:49.048 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0055055479999999999, 2024-04-25T20:10:49.048 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:49.048 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:49.048 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:49.048 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.048 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:49.049 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:49.049 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.049 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.049 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.049 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:49.049 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:49.049 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.049 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.049 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.049 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:49.049 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:49.049 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:49.049 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.049 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.050 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:49.050 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167766+0000", 2024-04-25T20:10:49.050 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.122e-06 2024-04-25T20:10:49.050 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.050 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.050 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:49.050 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167776+0000", 2024-04-25T20:10:49.050 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0515e-05 2024-04-25T20:10:49.050 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.050 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.050 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:49.050 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167782+0000", 2024-04-25T20:10:49.050 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1149999999999999e-06 2024-04-25T20:10:49.051 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.051 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.051 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:49.051 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167837+0000", 2024-04-25T20:10:49.051 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4763999999999999e-05 2024-04-25T20:10:49.051 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.051 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.051 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:49.051 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167855+0000", 2024-04-25T20:10:49.051 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7589e-05 2024-04-25T20:10:49.051 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.051 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.051 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:49.051 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173221+0000", 2024-04-25T20:10:49.052 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.005365903 2024-04-25T20:10:49.052 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.052 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.052 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:49.052 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173266+0000", 2024-04-25T20:10:49.052 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5059999999999999e-05 2024-04-25T20:10:49.052 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.052 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:49.052 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.052 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.052 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.052 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:32 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-04-25T20:10:49.052 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:49.052 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 91.873180852000004, 2024-04-25T20:10:49.052 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0051558239999999998, 2024-04-25T20:10:49.052 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:49.052 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:49.053 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:49.053 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.053 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:49.053 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:49.053 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.053 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.053 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.053 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:49.053 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:49.053 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.053 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.053 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.053 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:49.053 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:49.053 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:49.053 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.054 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.054 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:49.054 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168173+0000", 2024-04-25T20:10:49.054 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7663999999999997e-05 2024-04-25T20:10:49.054 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.054 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.054 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:49.054 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168182+0000", 2024-04-25T20:10:49.054 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0650000000000005e-06 2024-04-25T20:10:49.054 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.054 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.054 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:49.054 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168188+0000", 2024-04-25T20:10:49.055 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0589999999999999e-06 2024-04-25T20:10:49.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.055 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.055 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:49.055 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168239+0000", 2024-04-25T20:10:49.055 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1467e-05 2024-04-25T20:10:49.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.055 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.055 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:49.055 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168260+0000", 2024-04-25T20:10:49.055 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0531e-05 2024-04-25T20:10:49.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.055 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.055 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:49.055 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173268+0000", 2024-04-25T20:10:49.055 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0050087980000000001 2024-04-25T20:10:49.056 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.056 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.056 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:49.056 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173291+0000", 2024-04-25T20:10:49.056 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.268e-05 2024-04-25T20:10:49.056 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.056 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:49.056 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.056 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.056 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.056 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:33 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'19, mlcod=16'14)", 2024-04-25T20:10:49.056 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:10:49.056 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 91.872670432999996, 2024-04-25T20:10:49.056 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046697049999999997, 2024-04-25T20:10:49.056 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:49.056 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:49.057 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:49.057 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.057 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:49.057 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:10:49.057 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.057 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.057 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.057 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:49.057 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:10:49.057 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.057 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.057 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.057 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:49.058 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168645+0000", 2024-04-25T20:10:49.058 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:49.058 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.058 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.058 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:49.058 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168651+0000", 2024-04-25T20:10:49.058 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1079999999999998e-06 2024-04-25T20:10:49.058 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.058 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.058 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:49.058 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168660+0000", 2024-04-25T20:10:49.058 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0010000000000007e-06 2024-04-25T20:10:49.059 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.059 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.059 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:49.059 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168666+0000", 2024-04-25T20:10:49.059 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7780000000000004e-06 2024-04-25T20:10:49.059 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.059 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.059 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:49.059 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168701+0000", 2024-04-25T20:10:49.059 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4490999999999999e-05 2024-04-25T20:10:49.059 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.059 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.059 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:49.059 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168729+0000", 2024-04-25T20:10:49.059 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8476999999999999e-05 2024-04-25T20:10:49.059 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.060 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.060 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:49.060 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173294+0000", 2024-04-25T20:10:49.060 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0045650730000000002 2024-04-25T20:10:49.060 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.060 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.060 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:49.060 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173315+0000", 2024-04-25T20:10:49.060 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1245000000000001e-05 2024-04-25T20:10:49.060 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.060 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:49.060 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.060 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.060 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.060 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:34 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'20, mlcod=16'14)", 2024-04-25T20:10:49.060 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:10:49.061 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 91.872171503999994, 2024-04-25T20:10:49.061 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041943309999999999, 2024-04-25T20:10:49.061 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:49.061 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:49.061 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:49.061 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.061 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:49.061 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:10:49.061 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.061 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.061 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.061 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:49.061 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:10:49.061 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.061 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.061 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.062 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:49.062 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169144+0000", 2024-04-25T20:10:49.062 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:49.062 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.062 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.062 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:49.062 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169150+0000", 2024-04-25T20:10:49.062 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9150000000000001e-06 2024-04-25T20:10:49.062 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.062 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.062 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:49.062 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169159+0000", 2024-04-25T20:10:49.062 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8899999999999996e-06 2024-04-25T20:10:49.062 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.062 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.063 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:49.063 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169169+0000", 2024-04-25T20:10:49.063 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0441e-05 2024-04-25T20:10:49.063 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.063 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.063 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:49.063 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169206+0000", 2024-04-25T20:10:49.063 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6816000000000001e-05 2024-04-25T20:10:49.063 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.063 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.063 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:49.063 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169234+0000", 2024-04-25T20:10:49.063 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8008999999999999e-05 2024-04-25T20:10:49.063 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.063 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.063 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:49.064 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173317+0000", 2024-04-25T20:10:49.064 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0040830160000000001 2024-04-25T20:10:49.064 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.064 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.064 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:49.064 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173339+0000", 2024-04-25T20:10:49.064 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1897000000000001e-05 2024-04-25T20:10:49.064 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.064 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:49.064 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.064 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.064 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.064 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:35 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'14)", 2024-04-25T20:10:49.064 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:10:49.064 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 91.871679185000005, 2024-04-25T20:10:49.065 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037236529999999999, 2024-04-25T20:10:49.065 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:49.065 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:49.065 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:49.065 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.065 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:49.065 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:10:49.065 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.065 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.065 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.065 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:49.065 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:10:49.065 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.065 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.065 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.066 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:49.066 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169636+0000", 2024-04-25T20:10:49.066 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:49.066 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.066 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.066 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:49.066 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169678+0000", 2024-04-25T20:10:49.066 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1752000000000003e-05 2024-04-25T20:10:49.066 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.066 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.066 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:49.066 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169688+0000", 2024-04-25T20:10:49.066 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0098000000000001e-05 2024-04-25T20:10:49.066 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.067 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.067 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:49.067 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169694+0000", 2024-04-25T20:10:49.067 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2349999999999996e-06 2024-04-25T20:10:49.067 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.067 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.067 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:49.067 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169733+0000", 2024-04-25T20:10:49.067 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8481e-05 2024-04-25T20:10:49.067 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.067 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.067 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:49.067 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169750+0000", 2024-04-25T20:10:49.067 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7509000000000001e-05 2024-04-25T20:10:49.067 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.067 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.068 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:49.068 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173340+0000", 2024-04-25T20:10:49.068 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0035898449999999999 2024-04-25T20:10:49.068 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.068 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.068 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:49.068 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173361+0000", 2024-04-25T20:10:49.068 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.039e-05 2024-04-25T20:10:49.068 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.068 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:49.068 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.068 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.068 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.068 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:36 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'22, mlcod=16'14)", 2024-04-25T20:10:49.069 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:49.069 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 91.871179124999998, 2024-04-25T20:10:49.069 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032472120000000002, 2024-04-25T20:10:49.069 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:49.069 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:49.069 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:49.069 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.069 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:49.069 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:49.069 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.069 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.069 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.070 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:49.070 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:49.070 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.070 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.070 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.070 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:49.070 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:49.070 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:49.070 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.070 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.070 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:49.070 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170152+0000", 2024-04-25T20:10:49.071 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5798e-05 2024-04-25T20:10:49.071 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.071 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.071 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:49.071 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170161+0000", 2024-04-25T20:10:49.071 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.064e-06 2024-04-25T20:10:49.071 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.071 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.071 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:49.071 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170167+0000", 2024-04-25T20:10:49.071 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8390000000000003e-06 2024-04-25T20:10:49.071 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.071 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.071 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:49.071 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170202+0000", 2024-04-25T20:10:49.072 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.5120999999999998e-05 2024-04-25T20:10:49.072 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.072 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.072 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:49.072 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170219+0000", 2024-04-25T20:10:49.072 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6929999999999999e-05 2024-04-25T20:10:49.072 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.072 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.072 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:49.072 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173362+0000", 2024-04-25T20:10:49.072 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0031427619999999999 2024-04-25T20:10:49.072 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.072 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.072 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:49.072 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173384+0000", 2024-04-25T20:10:49.073 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2237000000000001e-05 2024-04-25T20:10:49.073 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.073 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:49.073 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.073 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.073 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.073 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:67 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'41, mlcod=16'39)", 2024-04-25T20:10:49.073 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:10:49.073 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 91.829379720999995, 2024-04-25T20:10:49.073 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064192549999999996, 2024-04-25T20:10:49.073 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:49.073 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:49.073 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:49.073 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.073 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:49.074 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:10:49.074 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.074 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.074 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.074 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:49.074 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:10:49.074 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.074 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.074 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.074 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:49.074 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211935+0000", 2024-04-25T20:10:49.074 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999981 2024-04-25T20:10:49.074 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.074 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.075 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:49.075 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211948+0000", 2024-04-25T20:10:49.075 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3531e-05 2024-04-25T20:10:49.075 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.075 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.075 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:49.075 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211967+0000", 2024-04-25T20:10:49.075 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8326999999999999e-05 2024-04-25T20:10:49.075 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.075 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.075 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:49.075 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211975+0000", 2024-04-25T20:10:49.075 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8589999999999994e-06 2024-04-25T20:10:49.075 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.076 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.076 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:49.076 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212020+0000", 2024-04-25T20:10:49.076 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4626999999999998e-05 2024-04-25T20:10:49.076 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.076 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.076 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:49.076 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212045+0000", 2024-04-25T20:10:49.076 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4508999999999999e-05 2024-04-25T20:10:49.076 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.076 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.076 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:49.076 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218316+0000", 2024-04-25T20:10:49.076 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0062711640000000001 2024-04-25T20:10:49.076 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.076 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.077 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:49.077 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218356+0000", 2024-04-25T20:10:49.077 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9993000000000001e-05 2024-04-25T20:10:49.077 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.077 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:49.077 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.077 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.077 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.077 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:68 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-04-25T20:10:49.077 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:49.077 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 91.828930706999998, 2024-04-25T20:10:49.077 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011231365, 2024-04-25T20:10:49.077 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:49.077 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:49.077 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:49.078 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.078 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:49.078 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:49.078 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.078 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.078 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.078 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:49.078 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:49.078 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.078 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.078 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.078 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:49.078 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:49.079 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:49.079 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.079 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.079 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:49.079 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212392+0000", 2024-04-25T20:10:49.079 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.79e-06 2024-04-25T20:10:49.079 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.079 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.079 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:49.079 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212406+0000", 2024-04-25T20:10:49.079 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3577000000000001e-05 2024-04-25T20:10:49.079 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.079 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.079 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:49.080 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212414+0000", 2024-04-25T20:10:49.080 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6510000000000001e-06 2024-04-25T20:10:49.080 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.080 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.080 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:49.080 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217114+0000", 2024-04-25T20:10:49.080 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047005379999999998 2024-04-25T20:10:49.080 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.080 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.080 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:49.080 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217141+0000", 2024-04-25T20:10:49.080 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6984000000000001e-05 2024-04-25T20:10:49.080 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.080 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.080 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:49.081 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223563+0000", 2024-04-25T20:10:49.081 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064216999999999998 2024-04-25T20:10:49.081 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.081 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.081 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:49.081 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223617+0000", 2024-04-25T20:10:49.081 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4011000000000002e-05 2024-04-25T20:10:49.081 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.081 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:49.081 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.081 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.081 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.081 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:69 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-04-25T20:10:49.081 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:10:49.081 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 91.828398516999997, 2024-04-25T20:10:49.082 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.010730094000000001, 2024-04-25T20:10:49.082 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:49.082 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:49.082 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:49.082 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.082 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:49.082 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:10:49.082 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.082 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.082 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.082 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:49.082 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:10:49.082 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.082 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.083 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.083 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:49.083 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212917+0000", 2024-04-25T20:10:49.083 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:49.083 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.083 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.083 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:49.083 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212957+0000", 2024-04-25T20:10:49.083 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0692000000000003e-05 2024-04-25T20:10:49.083 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.083 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.083 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:49.083 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212973+0000", 2024-04-25T20:10:49.083 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.575e-05 2024-04-25T20:10:49.083 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.084 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.084 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:49.084 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212980+0000", 2024-04-25T20:10:49.084 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1679999999999998e-06 2024-04-25T20:10:49.084 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.084 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.084 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:49.084 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217449+0000", 2024-04-25T20:10:49.084 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.004468367 2024-04-25T20:10:49.084 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.084 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.084 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:49.084 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217470+0000", 2024-04-25T20:10:49.084 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0749000000000001e-05 2024-04-25T20:10:49.084 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.085 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.085 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:49.085 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223621+0000", 2024-04-25T20:10:49.085 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0061512880000000004 2024-04-25T20:10:49.085 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.085 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.085 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:49.085 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223648+0000", 2024-04-25T20:10:49.085 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6961000000000002e-05 2024-04-25T20:10:49.085 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.085 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:49.085 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.085 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.085 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.086 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:70 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'44, mlcod=16'39)", 2024-04-25T20:10:49.086 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:49.086 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 91.827639789000003, 2024-04-25T20:10:49.086 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.013553358, 2024-04-25T20:10:49.086 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:49.086 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:49.086 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:49.086 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.086 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:49.086 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:49.086 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.086 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.086 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.086 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:49.087 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:49.087 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.087 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.087 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.087 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:49.087 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:49.087 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:49.087 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.087 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.087 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:49.087 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213682+0000", 2024-04-25T20:10:49.087 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3439999999999997e-06 2024-04-25T20:10:49.087 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.087 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.087 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:49.088 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213693+0000", 2024-04-25T20:10:49.088 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1049999999999999e-05 2024-04-25T20:10:49.088 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.088 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.088 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:49.088 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213699+0000", 2024-04-25T20:10:49.088 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9399999999999999e-06 2024-04-25T20:10:49.088 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.088 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.088 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:49.088 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218086+0000", 2024-04-25T20:10:49.088 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0043871240000000001 2024-04-25T20:10:49.088 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.088 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.088 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:49.089 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218106+0000", 2024-04-25T20:10:49.089 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.997e-05 2024-04-25T20:10:49.089 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.089 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.089 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:49.089 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.227186+0000", 2024-04-25T20:10:49.089 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.009079574 2024-04-25T20:10:49.089 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.089 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.089 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:49.089 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.227230+0000", 2024-04-25T20:10:49.089 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4147999999999998e-05 2024-04-25T20:10:49.089 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.089 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:49.089 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.089 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.090 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.090 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:106 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-04-25T20:10:49.090 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:10:49.090 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 91.456500688999995, 2024-04-25T20:10:49.090 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002936756, 2024-04-25T20:10:49.090 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:49.090 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:49.090 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:49.090 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.090 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:49.090 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:10:49.090 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.090 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.091 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.091 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:49.091 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:10:49.091 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.091 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.091 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.091 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:49.091 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584815+0000", 2024-04-25T20:10:49.091 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:49.091 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.091 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.091 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:49.091 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584860+0000", 2024-04-25T20:10:49.091 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5161999999999999e-05 2024-04-25T20:10:49.092 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.092 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.092 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:49.092 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584874+0000", 2024-04-25T20:10:49.092 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4464000000000001e-05 2024-04-25T20:10:49.092 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.092 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.092 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:49.092 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584881+0000", 2024-04-25T20:10:49.092 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.409e-06 2024-04-25T20:10:49.092 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.092 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.092 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:49.092 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584915+0000", 2024-04-25T20:10:49.092 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4007000000000002e-05 2024-04-25T20:10:49.092 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.093 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.093 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:49.093 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584933+0000", 2024-04-25T20:10:49.093 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7737999999999999e-05 2024-04-25T20:10:49.093 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.093 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.093 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:49.093 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587717+0000", 2024-04-25T20:10:49.093 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002784711 2024-04-25T20:10:49.093 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.093 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.093 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:49.093 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587752+0000", 2024-04-25T20:10:49.093 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4915e-05 2024-04-25T20:10:49.094 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.094 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:49.094 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.094 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.094 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.094 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:112 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'74, mlcod=16'66)", 2024-04-25T20:10:49.094 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:10:49.094 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 91.454242613999995, 2024-04-25T20:10:49.094 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00318724, 2024-04-25T20:10:49.094 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:49.094 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:49.094 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:49.094 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.094 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:49.095 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:10:49.095 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.095 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.095 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:49.095 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:10:49.095 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.095 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.095 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:49.095 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587073+0000", 2024-04-25T20:10:49.095 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:49.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.095 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.095 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:49.096 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587136+0000", 2024-04-25T20:10:49.096 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2928999999999993e-05 2024-04-25T20:10:49.096 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.096 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.096 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:49.096 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587147+0000", 2024-04-25T20:10:49.096 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1182999999999999e-05 2024-04-25T20:10:49.096 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.096 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.096 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:49.096 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587153+0000", 2024-04-25T20:10:49.096 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9059999999999999e-06 2024-04-25T20:10:49.096 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.096 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.096 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:49.096 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587306+0000", 2024-04-25T20:10:49.097 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000152848 2024-04-25T20:10:49.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.097 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.097 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:49.097 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587330+0000", 2024-04-25T20:10:49.097 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4304e-05 2024-04-25T20:10:49.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.097 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.097 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:49.097 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.590230+0000", 2024-04-25T20:10:49.097 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029004629999999998 2024-04-25T20:10:49.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.097 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.097 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:49.098 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.590261+0000", 2024-04-25T20:10:49.098 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0323000000000001e-05 2024-04-25T20:10:49.098 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.098 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:49.098 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.098 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.098 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.098 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:118 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'80, mlcod=16'78)", 2024-04-25T20:10:49.098 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:10:49.098 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 91.449456034999997, 2024-04-25T20:10:49.098 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029618959999999999, 2024-04-25T20:10:49.098 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:49.098 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:49.099 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:49.099 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.099 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:49.099 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:10:49.099 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.099 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.099 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.099 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:49.099 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:10:49.099 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.099 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.099 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.099 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:49.099 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591859+0000", 2024-04-25T20:10:49.100 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:49.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.100 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.100 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:49.100 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591918+0000", 2024-04-25T20:10:49.100 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8842999999999999e-05 2024-04-25T20:10:49.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.100 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.100 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:49.100 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591933+0000", 2024-04-25T20:10:49.100 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4600000000000001e-05 2024-04-25T20:10:49.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.100 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.100 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:49.100 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591939+0000", 2024-04-25T20:10:49.100 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4559999999999997e-06 2024-04-25T20:10:49.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.101 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.101 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:49.101 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591990+0000", 2024-04-25T20:10:49.101 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0318999999999997e-05 2024-04-25T20:10:49.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.101 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.101 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:49.101 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.592008+0000", 2024-04-25T20:10:49.101 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8819000000000001e-05 2024-04-25T20:10:49.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.101 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.101 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:49.101 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.594784+0000", 2024-04-25T20:10:49.101 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027750850000000001 2024-04-25T20:10:49.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.101 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.102 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:49.102 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.594822+0000", 2024-04-25T20:10:49.102 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8522000000000001e-05 2024-04-25T20:10:49.102 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.102 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:49.102 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.102 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.102 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:139 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'97, mlcod=17'87)", 2024-04-25T20:10:49.102 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:10:49.102 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 90.91541943, 2024-04-25T20:10:49.102 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048546290000000001, 2024-04-25T20:10:49.102 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:49.102 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:49.103 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:49.103 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.103 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:49.103 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:10:49.103 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.103 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.103 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.103 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:49.103 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:10:49.103 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.103 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.103 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.103 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:49.104 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125896+0000", 2024-04-25T20:10:49.104 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:49.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.104 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.104 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:49.104 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125908+0000", 2024-04-25T20:10:49.104 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.137e-05 2024-04-25T20:10:49.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.104 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.104 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:49.104 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125915+0000", 2024-04-25T20:10:49.104 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3939999999999998e-06 2024-04-25T20:10:49.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.104 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.104 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:49.104 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125922+0000", 2024-04-25T20:10:49.104 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1930000000000004e-06 2024-04-25T20:10:49.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.105 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.105 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:49.105 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125959+0000", 2024-04-25T20:10:49.105 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6701e-05 2024-04-25T20:10:49.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.105 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.105 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:49.105 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125972+0000", 2024-04-25T20:10:49.105 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2578000000000001e-05 2024-04-25T20:10:49.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.105 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.105 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:49.105 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130719+0000", 2024-04-25T20:10:49.105 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047475800000000004 2024-04-25T20:10:49.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.106 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.106 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:49.106 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130751+0000", 2024-04-25T20:10:49.106 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2283000000000002e-05 2024-04-25T20:10:49.106 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.106 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:49.106 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.106 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.106 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:140 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'98, mlcod=17'87)", 2024-04-25T20:10:49.106 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:49.106 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 90.914994078999996, 2024-04-25T20:10:49.106 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048310990000000002, 2024-04-25T20:10:49.106 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:49.106 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:49.107 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:49.107 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.107 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:49.107 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:49.107 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.107 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.107 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:49.107 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:49.107 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.107 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.107 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:49.107 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:49.107 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:49.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.108 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.108 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:49.108 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126427+0000", 2024-04-25T20:10:49.108 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000105572 2024-04-25T20:10:49.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.108 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.108 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:49.108 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126435+0000", 2024-04-25T20:10:49.108 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1920000000000005e-06 2024-04-25T20:10:49.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.108 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.108 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:49.108 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126439+0000", 2024-04-25T20:10:49.108 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6950000000000002e-06 2024-04-25T20:10:49.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.108 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.109 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:49.109 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126471+0000", 2024-04-25T20:10:49.109 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1699000000000003e-05 2024-04-25T20:10:49.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.109 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.109 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:49.109 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126484+0000", 2024-04-25T20:10:49.109 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.343e-05 2024-04-25T20:10:49.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.109 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.109 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:49.109 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131131+0000", 2024-04-25T20:10:49.109 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046471480000000003 2024-04-25T20:10:49.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.110 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.110 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:49.110 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131153+0000", 2024-04-25T20:10:49.110 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1753e-05 2024-04-25T20:10:49.110 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.110 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:49.110 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.110 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.110 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.110 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:141 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'99, mlcod=17'87)", 2024-04-25T20:10:49.110 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:49.110 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 90.914339135000006, 2024-04-25T20:10:49.110 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.004496847, 2024-04-25T20:10:49.111 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:49.111 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:49.111 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:49.111 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.111 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:49.111 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:49.111 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.111 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.111 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:49.111 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:49.111 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.111 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.111 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:49.111 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:49.112 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:49.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.112 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.112 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:49.112 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126980+0000", 2024-04-25T20:10:49.112 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.619e-06 2024-04-25T20:10:49.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.112 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.112 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:49.112 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126987+0000", 2024-04-25T20:10:49.112 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7220000000000002e-06 2024-04-25T20:10:49.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.112 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.112 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:49.112 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126992+0000", 2024-04-25T20:10:49.112 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8679999999999998e-06 2024-04-25T20:10:49.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.113 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.113 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:49.113 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130772+0000", 2024-04-25T20:10:49.113 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037803120000000001 2024-04-25T20:10:49.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.113 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.113 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:49.113 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130786+0000", 2024-04-25T20:10:49.113 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3592e-05 2024-04-25T20:10:49.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.113 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.113 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:49.113 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131451+0000", 2024-04-25T20:10:49.113 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000665206 2024-04-25T20:10:49.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.114 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.114 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:49.114 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131474+0000", 2024-04-25T20:10:49.114 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2957999999999999e-05 2024-04-25T20:10:49.114 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.114 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:49.114 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.114 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.114 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4982.0:9 1.7 e59/55 1:e33cb61b:::journal_data.1.145ee3bcd187.2:head v 59'1, mlcod=0'0)", 2024-04-25T20:10:49.114 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:48.400129+0000", 2024-04-25T20:10:49.114 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.64118701199999995, 2024-04-25T20:10:49.114 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0058907949999999999, 2024-04-25T20:10:49.115 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:49.115 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:49.115 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:49.115 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.115 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:49.115 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400129+0000", 2024-04-25T20:10:49.115 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.115 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.115 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:49.115 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400129+0000", 2024-04-25T20:10:49.115 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.115 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.115 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:49.116 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400128+0000", 2024-04-25T20:10:49.116 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999986 2024-04-25T20:10:49.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.116 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.116 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:49.116 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400143+0000", 2024-04-25T20:10:49.116 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5358000000000001e-05 2024-04-25T20:10:49.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.116 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.116 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:49.116 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400161+0000", 2024-04-25T20:10:49.116 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7280999999999999e-05 2024-04-25T20:10:49.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.116 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.116 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:49.116 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400170+0000", 2024-04-25T20:10:49.117 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.2299999999999997e-06 2024-04-25T20:10:49.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.117 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.117 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:49.117 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400231+0000", 2024-04-25T20:10:49.117 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0902000000000001e-05 2024-04-25T20:10:49.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.117 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.117 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:49.117 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400249+0000", 2024-04-25T20:10:49.117 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7977000000000001e-05 2024-04-25T20:10:49.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.117 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.117 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:49.117 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.405996+0000", 2024-04-25T20:10:49.117 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.005747355 2024-04-25T20:10:49.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.118 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.118 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:49.118 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.406020+0000", 2024-04-25T20:10:49.118 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3952e-05 2024-04-25T20:10:49.118 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.118 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:49.118 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.118 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.118 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4982.0:13 1.7 e59/55 1:e33cb61b:::journal_data.1.145ee3bcd187.2:head v 59'2, mlcod=0'0)", 2024-04-25T20:10:49.118 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:48.400761+0000", 2024-04-25T20:10:49.118 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.64055469600000003, 2024-04-25T20:10:49.118 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0057307879999999997, 2024-04-25T20:10:49.118 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:49.119 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:49.119 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:49.119 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.119 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:49.119 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400761+0000", 2024-04-25T20:10:49.119 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.119 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.119 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:49.119 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400761+0000", 2024-04-25T20:10:49.119 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.119 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.119 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:49.119 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400761+0000", 2024-04-25T20:10:49.120 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:49.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.120 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.120 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:49.120 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400770+0000", 2024-04-25T20:10:49.120 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7129999999999994e-06 2024-04-25T20:10:49.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.120 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.120 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:49.120 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400781+0000", 2024-04-25T20:10:49.120 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1255e-05 2024-04-25T20:10:49.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.120 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.120 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:49.120 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400789+0000", 2024-04-25T20:10:49.121 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8550000000000002e-06 2024-04-25T20:10:49.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.121 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.121 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:49.121 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400822+0000", 2024-04-25T20:10:49.121 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3340000000000003e-05 2024-04-25T20:10:49.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.121 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.121 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:49.121 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400834+0000", 2024-04-25T20:10:49.121 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1377e-05 2024-04-25T20:10:49.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.121 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.121 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:49.121 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.406459+0000", 2024-04-25T20:10:49.121 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0056257829999999997 2024-04-25T20:10:49.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.122 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.122 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:49.122 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.406492+0000", 2024-04-25T20:10:49.122 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2926999999999998e-05 2024-04-25T20:10:49.122 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.122 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:49.122 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.122 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.122 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4982.0:17 1.7 e59/55 1:e33cb61b:::journal_data.1.145ee3bcd187.2:head v 59'3, mlcod=0'0)", 2024-04-25T20:10:49.122 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:48.401348+0000", 2024-04-25T20:10:49.122 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.63996783899999998, 2024-04-25T20:10:49.122 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053994259999999997, 2024-04-25T20:10:49.122 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:49.123 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:49.123 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:49.123 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.123 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:49.123 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.401348+0000", 2024-04-25T20:10:49.123 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.123 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.123 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:49.123 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.401348+0000", 2024-04-25T20:10:49.123 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:49.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.123 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.124 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:49.124 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.401348+0000", 2024-04-25T20:10:49.124 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:49.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.124 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.124 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:49.124 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.401357+0000", 2024-04-25T20:10:49.124 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.1919999999999994e-06 2024-04-25T20:10:49.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.124 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.124 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:49.124 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.401365+0000", 2024-04-25T20:10:49.124 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6260000000000003e-06 2024-04-25T20:10:49.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.125 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.125 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:49.125 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.401370+0000", 2024-04-25T20:10:49.125 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5520000000000004e-06 2024-04-25T20:10:49.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.125 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.125 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:49.125 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.406055+0000", 2024-04-25T20:10:49.125 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046851050000000002 2024-04-25T20:10:49.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.125 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.125 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:49.125 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.406077+0000", 2024-04-25T20:10:49.125 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1078999999999999e-05 2024-04-25T20:10:49.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.126 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.126 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:49.126 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.406721+0000", 2024-04-25T20:10:49.126 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00064424999999999997 2024-04-25T20:10:49.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:49.126 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:49.126 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:49.126 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.406748+0000", 2024-04-25T20:10:49.126 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7012000000000002e-05 2024-04-25T20:10:49.126 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.126 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:49.126 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.126 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:49.126 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:49.126 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:49.128 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T20:10:49.128 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:49.128+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:49.213 INFO:tasks.workunit.client.0.smithi039.stderr:+ restore_commit_position 145ee3bcd187 2024-04-25T20:10:49.213 INFO:tasks.workunit.client.0.smithi039.stderr:+ local journal=145ee3bcd187 2024-04-25T20:10:49.213 INFO:tasks.workunit.client.0.smithi039.stderr:+ rados -p rbd setomapval journal.145ee3bcd187 client_ 2024-04-25T20:10:49.228 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:49.227+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:49.250 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd journal inspect --image testrbdjournal471061 --verbose 2024-04-25T20:10:49.250 INFO:tasks.workunit.client.0.smithi039.stderr:+ awk ' 2024-04-25T20:10:49.251 INFO:tasks.workunit.client.0.smithi039.stderr: /AioFlush/ {a++} # match: "event_type": "AioFlush", 2024-04-25T20:10:49.251 INFO:tasks.workunit.client.0.smithi039.stderr: /AioWrite/ {w++} # match: "event_type": "AioWrite", 2024-04-25T20:10:49.251 INFO:tasks.workunit.client.0.smithi039.stderr: /SnapCreate/ {s++} # match: "event_type": "SnapCreate", 2024-04-25T20:10:49.251 INFO:tasks.workunit.client.0.smithi039.stderr: /OpFinish/ {f++} # match: "event_type": "OpFinish", 2024-04-25T20:10:49.251 INFO:tasks.workunit.client.0.smithi039.stderr: /entries inspected/ {t=$1; e=$4} # match: 14 entries inspected, 0 errors 2024-04-25T20:10:49.251 INFO:tasks.workunit.client.0.smithi039.stderr: {print} # for diagnostic 2024-04-25T20:10:49.251 INFO:tasks.workunit.client.0.smithi039.stderr: END { 2024-04-25T20:10:49.251 INFO:tasks.workunit.client.0.smithi039.stderr: if (a != 2 || w != 10 || s != 1 || f != 1 || t != 14 || e != 0) exit(1) 2024-04-25T20:10:49.251 INFO:tasks.workunit.client.0.smithi039.stderr: } 2024-04-25T20:10:49.251 INFO:tasks.workunit.client.0.smithi039.stderr: ' 2024-04-25T20:10:49.279 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:49.279 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:49.279 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:49.279 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:49.289 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T20:10:49.310 INFO:tasks.workunit.client.0.smithi039.stderr:2024-04-25T20:10:49.309+0000 7f52b2e23640 -1 JournalMetadata: 0x55e9ac5e8300 failed to locate client: INSPECT 2024-04-25T20:10:49.312 INFO:tasks.workunit.client.0.smithi039.stdout:Entry: tag_id=1, commit_tid=1 2024-04-25T20:10:49.313 INFO:tasks.workunit.client.0.smithi039.stdout:{ 2024-04-25T20:10:49.313 INFO:tasks.workunit.client.0.smithi039.stdout: "event_type": "AioFlush", 2024-04-25T20:10:49.313 INFO:tasks.workunit.client.0.smithi039.stdout: "timestamp": "2024-04-25T20:10:47.663510+0000" 2024-04-25T20:10:49.313 INFO:tasks.workunit.client.0.smithi039.stdout:} 2024-04-25T20:10:49.313 INFO:tasks.workunit.client.0.smithi039.stdout:Entry: tag_id=1, commit_tid=2 2024-04-25T20:10:49.313 INFO:tasks.workunit.client.0.smithi039.stdout:{ 2024-04-25T20:10:49.313 INFO:tasks.workunit.client.0.smithi039.stdout: "event_type": "AioWrite", 2024-04-25T20:10:49.313 INFO:tasks.workunit.client.0.smithi039.stdout: "offset": 0, 2024-04-25T20:10:49.314 INFO:tasks.workunit.client.0.smithi039.stdout: "length": 4096, 2024-04-25T20:10:49.314 INFO:tasks.workunit.client.0.smithi039.stdout: "timestamp": "2024-04-25T20:10:47.665023+0000" 2024-04-25T20:10:49.314 INFO:tasks.workunit.client.0.smithi039.stdout:} 2024-04-25T20:10:49.314 INFO:tasks.workunit.client.0.smithi039.stdout:Entry: tag_id=1, commit_tid=3 2024-04-25T20:10:49.314 INFO:tasks.workunit.client.0.smithi039.stdout:{ 2024-04-25T20:10:49.314 INFO:tasks.workunit.client.0.smithi039.stdout: "event_type": "AioWrite", 2024-04-25T20:10:49.314 INFO:tasks.workunit.client.0.smithi039.stdout: "offset": 4096, 2024-04-25T20:10:49.314 INFO:tasks.workunit.client.0.smithi039.stdout: "length": 4096, 2024-04-25T20:10:49.314 INFO:tasks.workunit.client.0.smithi039.stdout: "timestamp": "2024-04-25T20:10:47.665085+0000" 2024-04-25T20:10:49.315 INFO:tasks.workunit.client.0.smithi039.stdout:} 2024-04-25T20:10:49.315 INFO:tasks.workunit.client.0.smithi039.stdout:Entry: tag_id=1, commit_tid=4 2024-04-25T20:10:49.315 INFO:tasks.workunit.client.0.smithi039.stdout:{ 2024-04-25T20:10:49.315 INFO:tasks.workunit.client.0.smithi039.stdout: "event_type": "AioWrite", 2024-04-25T20:10:49.315 INFO:tasks.workunit.client.0.smithi039.stdout: "offset": 8192, 2024-04-25T20:10:49.315 INFO:tasks.workunit.client.0.smithi039.stdout: "length": 4096, 2024-04-25T20:10:49.315 INFO:tasks.workunit.client.0.smithi039.stdout: "timestamp": "2024-04-25T20:10:47.665126+0000" 2024-04-25T20:10:49.315 INFO:tasks.workunit.client.0.smithi039.stdout:} 2024-04-25T20:10:49.316 INFO:tasks.workunit.client.0.smithi039.stdout:Entry: tag_id=1, commit_tid=5 2024-04-25T20:10:49.316 INFO:tasks.workunit.client.0.smithi039.stdout:{ 2024-04-25T20:10:49.316 INFO:tasks.workunit.client.0.smithi039.stdout: "event_type": "AioWrite", 2024-04-25T20:10:49.316 INFO:tasks.workunit.client.0.smithi039.stdout: "offset": 12288, 2024-04-25T20:10:49.316 INFO:tasks.workunit.client.0.smithi039.stdout: "length": 4096, 2024-04-25T20:10:49.316 INFO:tasks.workunit.client.0.smithi039.stdout: "timestamp": "2024-04-25T20:10:47.665196+0000" 2024-04-25T20:10:49.316 INFO:tasks.workunit.client.0.smithi039.stdout:} 2024-04-25T20:10:49.316 INFO:tasks.workunit.client.0.smithi039.stdout:Entry: tag_id=1, commit_tid=6 2024-04-25T20:10:49.316 INFO:tasks.workunit.client.0.smithi039.stdout:{ 2024-04-25T20:10:49.317 INFO:tasks.workunit.client.0.smithi039.stdout: "event_type": "AioWrite", 2024-04-25T20:10:49.317 INFO:tasks.workunit.client.0.smithi039.stdout: "offset": 16384, 2024-04-25T20:10:49.317 INFO:tasks.workunit.client.0.smithi039.stdout: "length": 4096, 2024-04-25T20:10:49.317 INFO:tasks.workunit.client.0.smithi039.stdout: "timestamp": "2024-04-25T20:10:47.665243+0000" 2024-04-25T20:10:49.317 INFO:tasks.workunit.client.0.smithi039.stdout:} 2024-04-25T20:10:49.317 INFO:tasks.workunit.client.0.smithi039.stdout:Entry: tag_id=1, commit_tid=7 2024-04-25T20:10:49.317 INFO:tasks.workunit.client.0.smithi039.stdout:{ 2024-04-25T20:10:49.317 INFO:tasks.workunit.client.0.smithi039.stdout: "event_type": "AioWrite", 2024-04-25T20:10:49.317 INFO:tasks.workunit.client.0.smithi039.stdout: "offset": 20480, 2024-04-25T20:10:49.318 INFO:tasks.workunit.client.0.smithi039.stdout: "length": 4096, 2024-04-25T20:10:49.318 INFO:tasks.workunit.client.0.smithi039.stdout: "timestamp": "2024-04-25T20:10:47.665304+0000" 2024-04-25T20:10:49.318 INFO:tasks.workunit.client.0.smithi039.stdout:} 2024-04-25T20:10:49.318 INFO:tasks.workunit.client.0.smithi039.stdout:Entry: tag_id=1, commit_tid=8 2024-04-25T20:10:49.318 INFO:tasks.workunit.client.0.smithi039.stdout:{ 2024-04-25T20:10:49.318 INFO:tasks.workunit.client.0.smithi039.stdout: "event_type": "AioWrite", 2024-04-25T20:10:49.318 INFO:tasks.workunit.client.0.smithi039.stdout: "offset": 24576, 2024-04-25T20:10:49.318 INFO:tasks.workunit.client.0.smithi039.stdout: "length": 4096, 2024-04-25T20:10:49.319 INFO:tasks.workunit.client.0.smithi039.stdout: "timestamp": "2024-04-25T20:10:47.665346+0000" 2024-04-25T20:10:49.319 INFO:tasks.workunit.client.0.smithi039.stdout:} 2024-04-25T20:10:49.319 INFO:tasks.workunit.client.0.smithi039.stdout:Entry: tag_id=1, commit_tid=9 2024-04-25T20:10:49.319 INFO:tasks.workunit.client.0.smithi039.stdout:{ 2024-04-25T20:10:49.319 INFO:tasks.workunit.client.0.smithi039.stdout: "event_type": "AioWrite", 2024-04-25T20:10:49.319 INFO:tasks.workunit.client.0.smithi039.stdout: "offset": 28672, 2024-04-25T20:10:49.319 INFO:tasks.workunit.client.0.smithi039.stdout: "length": 4096, 2024-04-25T20:10:49.319 INFO:tasks.workunit.client.0.smithi039.stdout: "timestamp": "2024-04-25T20:10:47.665387+0000" 2024-04-25T20:10:49.319 INFO:tasks.workunit.client.0.smithi039.stdout:} 2024-04-25T20:10:49.320 INFO:tasks.workunit.client.0.smithi039.stdout:Entry: tag_id=1, commit_tid=10 2024-04-25T20:10:49.320 INFO:tasks.workunit.client.0.smithi039.stdout:{ 2024-04-25T20:10:49.320 INFO:tasks.workunit.client.0.smithi039.stdout: "event_type": "AioWrite", 2024-04-25T20:10:49.320 INFO:tasks.workunit.client.0.smithi039.stdout: "offset": 32768, 2024-04-25T20:10:49.320 INFO:tasks.workunit.client.0.smithi039.stdout: "length": 4096, 2024-04-25T20:10:49.320 INFO:tasks.workunit.client.0.smithi039.stdout: "timestamp": "2024-04-25T20:10:47.665427+0000" 2024-04-25T20:10:49.320 INFO:tasks.workunit.client.0.smithi039.stdout:} 2024-04-25T20:10:49.320 INFO:tasks.workunit.client.0.smithi039.stdout:Entry: tag_id=1, commit_tid=11 2024-04-25T20:10:49.320 INFO:tasks.workunit.client.0.smithi039.stdout:{ 2024-04-25T20:10:49.321 INFO:tasks.workunit.client.0.smithi039.stdout: "event_type": "AioWrite", 2024-04-25T20:10:49.321 INFO:tasks.workunit.client.0.smithi039.stdout: "offset": 36864, 2024-04-25T20:10:49.321 INFO:tasks.workunit.client.0.smithi039.stdout: "length": 4096, 2024-04-25T20:10:49.321 INFO:tasks.workunit.client.0.smithi039.stdout: "timestamp": "2024-04-25T20:10:47.665468+0000" 2024-04-25T20:10:49.321 INFO:tasks.workunit.client.0.smithi039.stdout:} 2024-04-25T20:10:49.321 INFO:tasks.workunit.client.0.smithi039.stdout:Entry: tag_id=1, commit_tid=12 2024-04-25T20:10:49.321 INFO:tasks.workunit.client.0.smithi039.stdout:{ 2024-04-25T20:10:49.321 INFO:tasks.workunit.client.0.smithi039.stdout: "event_type": "AioFlush", 2024-04-25T20:10:49.321 INFO:tasks.workunit.client.0.smithi039.stdout: "timestamp": "2024-04-25T20:10:47.665495+0000" 2024-04-25T20:10:49.322 INFO:tasks.workunit.client.0.smithi039.stdout:} 2024-04-25T20:10:49.322 INFO:tasks.workunit.client.0.smithi039.stdout:Entry: tag_id=1, commit_tid=13 2024-04-25T20:10:49.322 INFO:tasks.workunit.client.0.smithi039.stdout:{ 2024-04-25T20:10:49.322 INFO:tasks.workunit.client.0.smithi039.stdout: "event_type": "SnapCreate", 2024-04-25T20:10:49.322 INFO:tasks.workunit.client.0.smithi039.stdout: "op_tid": 1, 2024-04-25T20:10:49.322 INFO:tasks.workunit.client.0.smithi039.stdout: "snap_name": "test", 2024-04-25T20:10:49.322 INFO:tasks.workunit.client.0.smithi039.stdout: "snapshot_namespace_type": "user", 2024-04-25T20:10:49.322 INFO:tasks.workunit.client.0.smithi039.stdout: "timestamp": "2024-04-25T20:10:48.468601+0000" 2024-04-25T20:10:49.322 INFO:tasks.workunit.client.0.smithi039.stdout:} 2024-04-25T20:10:49.323 INFO:tasks.workunit.client.0.smithi039.stdout:Entry: tag_id=1, commit_tid=14 2024-04-25T20:10:49.323 INFO:tasks.workunit.client.0.smithi039.stdout:{ 2024-04-25T20:10:49.323 INFO:tasks.workunit.client.0.smithi039.stdout: "event_type": "OpFinish", 2024-04-25T20:10:49.323 INFO:tasks.workunit.client.0.smithi039.stdout: "op_tid": 1, 2024-04-25T20:10:49.323 INFO:tasks.workunit.client.0.smithi039.stdout: "op_tid": 1, 2024-04-25T20:10:49.323 INFO:tasks.workunit.client.0.smithi039.stdout: "result": 0, 2024-04-25T20:10:49.323 INFO:tasks.workunit.client.0.smithi039.stdout: "timestamp": "2024-04-25T20:10:48.997560+0000" 2024-04-25T20:10:49.323 INFO:tasks.workunit.client.0.smithi039.stdout:} 2024-04-25T20:10:49.323 INFO:tasks.workunit.client.0.smithi039.stdout:Summary: 2024-04-25T20:10:49.324 INFO:tasks.workunit.client.0.smithi039.stdout: 14 entries inspected, 0 errors 2024-04-25T20:10:49.324 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd export testrbdjournal471061@test /tmp/rbd_journal47106/testrbdjournal471061.export 2024-04-25T20:10:49.329 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:49.329+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:49.389 INFO:tasks.workunit.client.0.smithi039.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-25T20:10:49.393 INFO:tasks.workunit.client.0.smithi039.stderr:+ cmp /tmp/rbd_journal47106/testrbdjournal47106.export /tmp/rbd_journal47106/testrbdjournal471061.export 2024-04-25T20:10:49.430 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:49.429+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:49.430 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:49.430 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:49.431 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:49.431 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:49.431 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:49.441 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T20:10:49.492 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd journal reset 134bdebc9a96 2024-04-25T20:10:49.530 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:49.529+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:49.570 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd journal inspect --verbose 134bdebc9a96 2024-04-25T20:10:49.570 INFO:tasks.workunit.client.0.smithi039.stderr:+ expect_false grep event_type 2024-04-25T20:10:49.570 INFO:tasks.workunit.client.0.smithi039.stderr:+ set -x 2024-04-25T20:10:49.570 INFO:tasks.workunit.client.0.smithi039.stderr:+ grep event_type 2024-04-25T20:10:49.593 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:49.594 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:49.594 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:49.594 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:10:49.594 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:49.603 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T20:10:49.611 INFO:tasks.workunit.client.0.smithi039.stderr:2024-04-25T20:10:49.611+0000 7f25a6523640 -1 JournalMetadata: 0x55e3fd286e00 failed to locate client: INSPECT 2024-04-25T20:10:49.614 INFO:tasks.workunit.client.0.smithi039.stderr:+ return 0 2024-04-25T20:10:49.614 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd snap purge testrbdjournal471061 2024-04-25T20:10:49.631 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:49.631+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:49.719 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:49.719 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:49.720 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:49.720 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:49.728 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T20:10:49.732 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:49.731+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:49.833 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:49.833+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:49.858 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T20:10:49.874 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:49.874 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:49.874 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:49.874 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:49.874 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:49.883 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T20:10:49.967 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:49.967+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:49.981 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:10:49.981 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:10:49.998 INFO:tasks.workunit.client.0.smithi039.stderr: Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2024-04-25T20:10:50.006 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd remove testrbdjournal471061 2024-04-25T20:10:50.015 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:10:50.017 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:10:50.035 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:50.034+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:50.048 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:10:50.048 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:10:50.057 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:50.057 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:50.057 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:50.057 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:10:50.057 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:50.067 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T20:10:50.067 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:10:50.067 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:10:50.084 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:10:50.085 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:10:50.090 INFO:tasks.workunit.client.0.smithi039.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete...2024-04-25T20:10:50.089+0000 7f7ae4f25640 0 -- 172.21.15.39:0/3661514706 >> [v2:172.21.15.39:6808/31831,v1:172.21.15.39:6809/31831] conn(0x55a1d97d5d20 msgr2=0x55a1d97a8b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-25T20:10:50.096 INFO:tasks.workunit.client.0.smithi039.stderr: Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete...2024-04-25T20:10:50.096+0000 7f7ae4f25640 0 -- 172.21.15.39:0/3661514706 >> [v2:172.21.15.135:6816/31737,v1:172.21.15.135:6817/31737] conn(0x55a1d975ad20 msgr2=0x55a1d975d110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-25T20:10:50.106 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:10:50.106 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:10:50.116 INFO:tasks.workunit.client.0.smithi039.stderr: Removing image: 100% complete...done. 2024-04-25T20:10:50.119 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd remove testrbdjournal47106 2024-04-25T20:10:50.125 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:10:50.125 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:10:50.135 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:50.135+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:50.142 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:10:50.143 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:10:50.186 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:50.186 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:50.195 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T20:10:50.201 INFO:tasks.workunit.client.0.smithi039.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete...2024-04-25T20:10:50.200+0000 7f312cf4e640 0 -- 172.21.15.39:0/3201646797 >> [v2:172.21.15.135:6816/31737,v1:172.21.15.135:6817/31737] conn(0x55aa43cb7ba0 msgr2=0x55aa43cb9f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-25T20:10:50.207 INFO:tasks.workunit.client.0.smithi039.stderr: Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete...2024-04-25T20:10:50.207+0000 7f312cf4e640 0 -- 172.21.15.39:0/3201646797 >> [v2:172.21.15.135:6816/31737,v1:172.21.15.135:6817/31737] conn(0x7f310c0704d0 msgr2=0x7f310c0728c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-25T20:10:50.236 INFO:tasks.workunit.client.0.smithi039.stderr: Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-25T20:10:50.237 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:50.236+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:50.240 INFO:tasks.workunit.client.0.smithi039.stderr:+ set +x 2024-04-25T20:10:50.240 INFO:tasks.workunit.client.0.smithi039.stderr:+ test_rbd_create 2024-04-25T20:10:50.240 INFO:tasks.workunit.client.0.smithi039.stderr:+ local image=testrbdcreate47106 2024-04-25T20:10:50.240 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd create --image-feature exclusive-lock --image-feature journaling --journal-pool rbd --journal-object-size 20M --journal-splay-width 6 --size 256 testrbdcreate47106 2024-04-25T20:10:50.302 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd_assert_eq testrbdcreate47106 'journal info' //journal/order 25 2024-04-25T20:10:50.302 INFO:tasks.workunit.client.0.smithi039.stderr:+ local image=testrbdcreate47106 2024-04-25T20:10:50.302 INFO:tasks.workunit.client.0.smithi039.stderr:+ local 'cmd=journal info' 2024-04-25T20:10:50.302 INFO:tasks.workunit.client.0.smithi039.stderr:+ local param=//journal/order 2024-04-25T20:10:50.302 INFO:tasks.workunit.client.0.smithi039.stderr:+ local expected_val=25 2024-04-25T20:10:50.302 INFO:tasks.workunit.client.0.smithi039.stderr:++ rbd --format xml journal info --image testrbdcreate47106 2024-04-25T20:10:50.302 INFO:tasks.workunit.client.0.smithi039.stderr:++ xmlstarlet sel -t -v //journal/order 2024-04-25T20:10:50.337 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:50.337+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:50.340 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:50.340 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:50.349 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T20:10:50.355 INFO:tasks.workunit.client.0.smithi039.stderr:+ local val=25 2024-04-25T20:10:50.355 INFO:tasks.workunit.client.0.smithi039.stderr:+ test 25 = 25 2024-04-25T20:10:50.355 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd_assert_eq testrbdcreate47106 'journal info' //journal/splay_width 6 2024-04-25T20:10:50.355 INFO:tasks.workunit.client.0.smithi039.stderr:+ local image=testrbdcreate47106 2024-04-25T20:10:50.355 INFO:tasks.workunit.client.0.smithi039.stderr:+ local 'cmd=journal info' 2024-04-25T20:10:50.355 INFO:tasks.workunit.client.0.smithi039.stderr:+ local param=//journal/splay_width 2024-04-25T20:10:50.355 INFO:tasks.workunit.client.0.smithi039.stderr:+ local expected_val=6 2024-04-25T20:10:50.355 INFO:tasks.workunit.client.0.smithi039.stderr:++ rbd --format xml journal info --image testrbdcreate47106 2024-04-25T20:10:50.356 INFO:tasks.workunit.client.0.smithi039.stderr:++ xmlstarlet sel -t -v //journal/splay_width 2024-04-25T20:10:50.407 INFO:tasks.workunit.client.0.smithi039.stderr:+ local val=6 2024-04-25T20:10:50.407 INFO:tasks.workunit.client.0.smithi039.stderr:+ test 6 = 6 2024-04-25T20:10:50.407 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd_assert_eq testrbdcreate47106 'journal info' //journal/object_pool rbd 2024-04-25T20:10:50.408 INFO:tasks.workunit.client.0.smithi039.stderr:+ local image=testrbdcreate47106 2024-04-25T20:10:50.408 INFO:tasks.workunit.client.0.smithi039.stderr:+ local 'cmd=journal info' 2024-04-25T20:10:50.408 INFO:tasks.workunit.client.0.smithi039.stderr:+ local param=//journal/object_pool 2024-04-25T20:10:50.408 INFO:tasks.workunit.client.0.smithi039.stderr:+ local expected_val=rbd 2024-04-25T20:10:50.408 INFO:tasks.workunit.client.0.smithi039.stderr:++ rbd --format xml journal info --image testrbdcreate47106 2024-04-25T20:10:50.408 INFO:tasks.workunit.client.0.smithi039.stderr:++ xmlstarlet sel -t -v //journal/object_pool 2024-04-25T20:10:50.438 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:50.437+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:50.458 INFO:tasks.workunit.client.0.smithi039.stderr:+ local val=rbd 2024-04-25T20:10:50.458 INFO:tasks.workunit.client.0.smithi039.stderr:+ test rbd = rbd 2024-04-25T20:10:50.458 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd remove testrbdcreate47106 2024-04-25T20:10:50.492 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:50.492 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:50.501 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T20:10:50.534 INFO:tasks.workunit.client.0.smithi039.stderr: Removing image: 1% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 12% complete... Removing image: 14% complete...2024-04-25T20:10:50.534+0000 7f82eb312640 0 -- 172.21.15.39:0/171474509 >> [v2:172.21.15.135:6800/31724,v1:172.21.15.135:6801/31724] conn(0x7f82c806e3b0 msgr2=0x7f82c806e830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-25T20:10:50.538 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:50.538+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:50.553 INFO:tasks.workunit.client.0.smithi039.stderr: Removing image: 15% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 23% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 37% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 48% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 53% complete... Removing image: 54% complete...2024-04-25T20:10:50.552+0000 7f82eb312640 0 -- 172.21.15.39:0/171474509 >> [v2:172.21.15.39:6816/31832,v1:172.21.15.39:6818/31832] conn(0x561134ea05a0 msgr2=0x561134ea2a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-25T20:10:50.639 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:50.640+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:50.646 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:50.646 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:50.655 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T20:10:50.741 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:50.740+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:50.758 INFO:tasks.workunit.client.0.smithi039.stderr: Removing image: 56% complete... Removing image: 57% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 62% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 73% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 87% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 98% complete... Removing image: 100% complete...done. 2024-04-25T20:10:50.760 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:50.760 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:50.761 INFO:tasks.workunit.client.0.smithi039.stderr:+ set +x 2024-04-25T20:10:50.761 INFO:tasks.workunit.client.0.smithi039.stderr:+ test_rbd_copy 2024-04-25T20:10:50.761 INFO:tasks.workunit.client.0.smithi039.stderr:+ local src=testrbdcopys47106 2024-04-25T20:10:50.761 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd create --size 256 testrbdcopys47106 2024-04-25T20:10:50.770 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T20:10:50.807 INFO:tasks.workunit.client.0.smithi039.stderr:+ local image=testrbdcopy47106 2024-04-25T20:10:50.808 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd copy --image-feature exclusive-lock --image-feature journaling --journal-pool rbd --journal-object-size 20M --journal-splay-width 6 testrbdcopys47106 testrbdcopy47106 2024-04-25T20:10:50.841 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:50.841+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:50.890 INFO:tasks.workunit.client.0.smithi039.stderr: Image copy: 1% complete... Image copy: 3% complete... Image copy: 4% complete... Image copy: 6% complete... Image copy: 7% complete... Image copy: 9% complete... Image copy: 10% complete... Image copy: 12% complete... Image copy: 14% complete... Image copy: 15% complete... Image copy: 17% complete... Image copy: 18% complete... Image copy: 20% complete... Image copy: 21% complete... Image copy: 23% complete... Image copy: 25% complete... Image copy: 26% complete... Image copy: 28% complete... Image copy: 29% complete... Image copy: 31% complete... Image copy: 32% complete... Image copy: 34% complete... Image copy: 35% complete... Image copy: 37% complete... Image copy: 39% complete... Image copy: 40% complete... Image copy: 42% complete... Image copy: 43% complete... Image copy: 45% complete... Image copy: 46% complete... Image copy: 48% complete... Image copy: 50% complete... Image copy: 51% complete... Image copy: 53% complete... Image copy: 54% complete... Image copy: 56% complete... Image copy: 57% complete... Image copy: 59% complete... Image copy: 60% complete... Image copy: 62% complete... Image copy: 64% complete... Image copy: 65% complete... Image copy: 67% complete... Image copy: 68% complete... Image copy: 70% complete...2024-04-25T20:10:50.890+0000 7f2fa2452640 0 -- 172.21.15.39:0/2219016662 >> [v2:172.21.15.135:6816/31737,v1:172.21.15.135:6817/31737] conn(0x7f2f84004310 msgr2=0x7f2f84004790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-25T20:10:50.894 INFO:tasks.workunit.client.0.smithi039.stderr: Image copy: 71% complete... Image copy: 73% complete... Image copy: 75% complete... Image copy: 76% complete... Image copy: 78% complete... Image copy: 79% complete... Image copy: 81% complete... Image copy: 82% complete... Image copy: 84% complete... Image copy: 85% complete... Image copy: 87% complete... Image copy: 89% complete... Image copy: 90% complete... Image copy: 92% complete... Image copy: 93% complete... Image copy: 95% complete... Image copy: 96% complete... Image copy: 98% complete...2024-04-25T20:10:50.894+0000 7f2fa3eb5640 0 -- 172.21.15.39:0/2219016662 >> [v2:172.21.15.39:6816/31832,v1:172.21.15.39:6818/31832] conn(0x55f97700c230 msgr2=0x55f97700e620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-25T20:10:50.897 INFO:tasks.workunit.client.0.smithi039.stderr: Image copy: 100% complete... Image copy: 100% complete...done. 2024-04-25T20:10:50.901 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd remove testrbdcopys47106 2024-04-25T20:10:50.917 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:50.917 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:50.926 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T20:10:50.942 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:50.942+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:50.970 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7, 2] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-25T20:10:50.970 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-25T20:10:50.970 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-04-25T20:10:50.970 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-04-25T20:10:50.976 INFO:tasks.workunit.client.0.smithi039.stderr:2024-04-25T20:10:50.976+0000 7f0f4058d640 0 -- 172.21.15.39:0/381645957 >> [v2:172.21.15.135:6808/31725,v1:172.21.15.135:6809/31725] conn(0x7f0f1c008be0 msgr2=0x7f0f1c00b0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-25T20:10:50.994 INFO:tasks.workunit.client.0.smithi039.stderr: Removing image: 1% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 12% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 23% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 37% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 48% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 62% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 73% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 87% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 98% complete...2024-04-25T20:10:50.994+0000 7f0f4058d640 0 -- 172.21.15.39:0/381645957 >> [v2:172.21.15.135:6808/31725,v1:172.21.15.135:6809/31725] conn(0x7f0f240704a0 msgr2=0x7f0f24072890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-25T20:10:51.008 INFO:tasks.workunit.client.0.smithi039.stderr: Removing image: 100% complete...done. 2024-04-25T20:10:51.008 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-25T20:10:51.011 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd_assert_eq testrbdcopy47106 'journal info' //journal/order 25 2024-04-25T20:10:51.011 INFO:tasks.workunit.client.0.smithi039.stderr:+ local image=testrbdcopy47106 2024-04-25T20:10:51.011 INFO:tasks.workunit.client.0.smithi039.stderr:+ local 'cmd=journal info' 2024-04-25T20:10:51.011 INFO:tasks.workunit.client.0.smithi039.stderr:+ local param=//journal/order 2024-04-25T20:10:51.012 INFO:tasks.workunit.client.0.smithi039.stderr:+ local expected_val=25 2024-04-25T20:10:51.012 INFO:tasks.workunit.client.0.smithi039.stderr:++ rbd --format xml journal info --image testrbdcopy47106 2024-04-25T20:10:51.012 INFO:tasks.workunit.client.0.smithi039.stderr:++ xmlstarlet sel -t -v //journal/order 2024-04-25T20:10:51.034 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:51.034 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:51.043 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T20:10:51.044 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:51.043+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:51.070 INFO:tasks.workunit.client.0.smithi039.stderr:+ local val=25 2024-04-25T20:10:51.070 INFO:tasks.workunit.client.0.smithi039.stderr:+ test 25 = 25 2024-04-25T20:10:51.070 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd_assert_eq testrbdcopy47106 'journal info' //journal/splay_width 6 2024-04-25T20:10:51.070 INFO:tasks.workunit.client.0.smithi039.stderr:+ local image=testrbdcopy47106 2024-04-25T20:10:51.070 INFO:tasks.workunit.client.0.smithi039.stderr:+ local 'cmd=journal info' 2024-04-25T20:10:51.070 INFO:tasks.workunit.client.0.smithi039.stderr:+ local param=//journal/splay_width 2024-04-25T20:10:51.070 INFO:tasks.workunit.client.0.smithi039.stderr:+ local expected_val=6 2024-04-25T20:10:51.070 INFO:tasks.workunit.client.0.smithi039.stderr:++ rbd --format xml journal info --image testrbdcopy47106 2024-04-25T20:10:51.070 INFO:tasks.workunit.client.0.smithi039.stderr:++ xmlstarlet sel -t -v //journal/splay_width 2024-04-25T20:10:51.129 INFO:tasks.workunit.client.0.smithi039.stderr:+ local val=6 2024-04-25T20:10:51.129 INFO:tasks.workunit.client.0.smithi039.stderr:+ test 6 = 6 2024-04-25T20:10:51.129 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd_assert_eq testrbdcopy47106 'journal info' //journal/object_pool rbd 2024-04-25T20:10:51.129 INFO:tasks.workunit.client.0.smithi039.stderr:+ local image=testrbdcopy47106 2024-04-25T20:10:51.130 INFO:tasks.workunit.client.0.smithi039.stderr:+ local 'cmd=journal info' 2024-04-25T20:10:51.130 INFO:tasks.workunit.client.0.smithi039.stderr:+ local param=//journal/object_pool 2024-04-25T20:10:51.130 INFO:tasks.workunit.client.0.smithi039.stderr:+ local expected_val=rbd 2024-04-25T20:10:51.130 INFO:tasks.workunit.client.0.smithi039.stderr:++ rbd --format xml journal info --image testrbdcopy47106 2024-04-25T20:10:51.130 INFO:tasks.workunit.client.0.smithi039.stderr:++ xmlstarlet sel -t -v //journal/object_pool 2024-04-25T20:10:51.144 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:51.144+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:51.146 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:51.146 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:51.156 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T20:10:51.185 INFO:tasks.workunit.client.0.smithi039.stderr:+ local val=rbd 2024-04-25T20:10:51.185 INFO:tasks.workunit.client.0.smithi039.stderr:+ test rbd = rbd 2024-04-25T20:10:51.185 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd remove testrbdcopy47106 2024-04-25T20:10:51.245 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:51.244+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:51.258 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:51.258 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:51.268 INFO:tasks.workunit.client.0.smithi039.stderr: Removing image: 1% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 12% complete... Removing image: 14% complete...2024-04-25T20:10:51.266+0000 7f69b8771640 0 -- 172.21.15.39:0/686920400 >> [v2:172.21.15.135:6816/31737,v1:172.21.15.135:6817/31737] conn(0x55c74c3226d0 msgr2=0x55c74c3457c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-25T20:10:51.269 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T20:10:51.274 INFO:tasks.workunit.client.0.smithi039.stderr: Removing image: 15% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 23% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 37% complete... Removing image: 39% complete...2024-04-25T20:10:51.272+0000 7f69b8771640 0 -- 172.21.15.39:0/686920400 >> [v2:172.21.15.135:6816/31737,v1:172.21.15.135:6817/31737] conn(0x7f699c0704a0 msgr2=0x7f699c072890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-25T20:10:51.304 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:10:51.304 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:51.304 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 61, 2024-04-25T20:10:51.304 INFO:teuthology.orchestra.run.smithi039.stdout: "fsid": "975ceafd-555d-4f71-81e4-1f70364da00f", 2024-04-25T20:10:51.304 INFO:teuthology.orchestra.run.smithi039.stdout: "created": "2024-04-25T20:09:03.377320+0000", 2024-04-25T20:10:51.304 INFO:teuthology.orchestra.run.smithi039.stdout: "modified": "2024-04-25T20:10:49.982724+0000", 2024-04-25T20:10:51.305 INFO:teuthology.orchestra.run.smithi039.stdout: "last_up_change": "2024-04-25T20:09:13.740341+0000", 2024-04-25T20:10:51.305 INFO:teuthology.orchestra.run.smithi039.stdout: "last_in_change": "2024-04-25T20:10:45.945659+0000", 2024-04-25T20:10:51.305 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T20:10:51.305 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_num": 5799936, 2024-04-25T20:10:51.305 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_set": [ 2024-04-25T20:10:51.305 INFO:teuthology.orchestra.run.smithi039.stdout: "pglog_hardlimit", 2024-04-25T20:10:51.305 INFO:teuthology.orchestra.run.smithi039.stdout: "purged_snapdirs", 2024-04-25T20:10:51.305 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_deletes", 2024-04-25T20:10:51.305 INFO:teuthology.orchestra.run.smithi039.stdout: "sortbitwise" 2024-04-25T20:10:51.305 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-04-25T20:10:51.305 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_version": 4, 2024-04-25T20:10:51.305 INFO:teuthology.orchestra.run.smithi039.stdout: "full_ratio": 0.94999998807907104, 2024-04-25T20:10:51.305 INFO:teuthology.orchestra.run.smithi039.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-04-25T20:10:51.305 INFO:teuthology.orchestra.run.smithi039.stdout: "nearfull_ratio": 0.85000002384185791, 2024-04-25T20:10:51.306 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_snapshot": "", 2024-04-25T20:10:51.306 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_max": 2, 2024-04-25T20:10:51.306 INFO:teuthology.orchestra.run.smithi039.stdout: "max_osd": 8, 2024-04-25T20:10:51.306 INFO:teuthology.orchestra.run.smithi039.stdout: "require_min_compat_client": "luminous", 2024-04-25T20:10:51.306 INFO:teuthology.orchestra.run.smithi039.stdout: "min_compat_client": "luminous", 2024-04-25T20:10:51.306 INFO:teuthology.orchestra.run.smithi039.stdout: "require_osd_release": "reef", 2024-04-25T20:10:51.306 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_crimson": false, 2024-04-25T20:10:51.306 INFO:teuthology.orchestra.run.smithi039.stdout: "pools": [ 2024-04-25T20:10:51.306 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.306 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 1, 2024-04-25T20:10:51.306 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": "rbd", 2024-04-25T20:10:51.306 INFO:teuthology.orchestra.run.smithi039.stdout: "create_time": "2024-04-25T20:09:14.202057+0000", 2024-04-25T20:10:51.306 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 8193, 2024-04-25T20:10:51.306 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-04-25T20:10:51.307 INFO:teuthology.orchestra.run.smithi039.stdout: "type": 1, 2024-04-25T20:10:51.307 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 2, 2024-04-25T20:10:51.307 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": 1, 2024-04-25T20:10:51.307 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_rule": 0, 2024-04-25T20:10:51.307 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_count": 0, 2024-04-25T20:10:51.307 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_target": 0, 2024-04-25T20:10:51.307 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T20:10:51.307 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T20:10:51.307 INFO:teuthology.orchestra.run.smithi039.stdout: "object_hash": 2, 2024-04-25T20:10:51.307 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscale_mode": "off", 2024-04-25T20:10:51.307 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": 8, 2024-04-25T20:10:51.307 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num": 8, 2024-04-25T20:10:51.307 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num_target": 8, 2024-04-25T20:10:51.307 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_target": 8, 2024-04-25T20:10:51.307 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_pending": 8, 2024-04-25T20:10:51.308 INFO:teuthology.orchestra.run.smithi039.stdout: "last_pg_merge_meta": { 2024-04-25T20:10:51.308 INFO:teuthology.orchestra.run.smithi039.stdout: "source_pgid": "0.0", 2024-04-25T20:10:51.308 INFO:teuthology.orchestra.run.smithi039.stdout: "ready_epoch": 0, 2024-04-25T20:10:51.308 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_started": 0, 2024-04-25T20:10:51.308 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_clean": 0, 2024-04-25T20:10:51.308 INFO:teuthology.orchestra.run.smithi039.stdout: "source_version": "0'0", 2024-04-25T20:10:51.308 INFO:teuthology.orchestra.run.smithi039.stdout: "target_version": "0'0" 2024-04-25T20:10:51.308 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.308 INFO:teuthology.orchestra.run.smithi039.stdout: "last_change": "61", 2024-04-25T20:10:51.308 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend": "0", 2024-04-25T20:10:51.308 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T20:10:51.308 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T20:10:51.308 INFO:teuthology.orchestra.run.smithi039.stdout: "auid": 0, 2024-04-25T20:10:51.308 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_mode": "selfmanaged", 2024-04-25T20:10:51.309 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_seq": 5, 2024-04-25T20:10:51.309 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_epoch": 61, 2024-04-25T20:10:51.309 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_snaps": [], 2024-04-25T20:10:51.309 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps": "[]", 2024-04-25T20:10:51.309 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_bytes": 0, 2024-04-25T20:10:51.309 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_objects": 0, 2024-04-25T20:10:51.309 INFO:teuthology.orchestra.run.smithi039.stdout: "tiers": [], 2024-04-25T20:10:51.309 INFO:teuthology.orchestra.run.smithi039.stdout: "tier_of": -1, 2024-04-25T20:10:51.309 INFO:teuthology.orchestra.run.smithi039.stdout: "read_tier": -1, 2024-04-25T20:10:51.309 INFO:teuthology.orchestra.run.smithi039.stdout: "write_tier": -1, 2024-04-25T20:10:51.309 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_mode": "none", 2024-04-25T20:10:51.309 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_bytes": 0, 2024-04-25T20:10:51.309 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_objects": 0, 2024-04-25T20:10:51.309 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T20:10:51.310 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T20:10:51.310 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T20:10:51.310 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_flush_age": 0, 2024-04-25T20:10:51.310 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_evict_age": 0, 2024-04-25T20:10:51.310 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profile": "", 2024-04-25T20:10:51.310 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_params": { 2024-04-25T20:10:51.310 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "none" 2024-04-25T20:10:51.310 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.310 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_period": 0, 2024-04-25T20:10:51.310 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_count": 0, 2024-04-25T20:10:51.310 INFO:teuthology.orchestra.run.smithi039.stdout: "use_gmt_hitset": true, 2024-04-25T20:10:51.310 INFO:teuthology.orchestra.run.smithi039.stdout: "min_read_recency_for_promote": 0, 2024-04-25T20:10:51.310 INFO:teuthology.orchestra.run.smithi039.stdout: "min_write_recency_for_promote": 0, 2024-04-25T20:10:51.310 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T20:10:51.311 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_search_last_n": 0, 2024-04-25T20:10:51.311 INFO:teuthology.orchestra.run.smithi039.stdout: "grade_table": [], 2024-04-25T20:10:51.311 INFO:teuthology.orchestra.run.smithi039.stdout: "stripe_width": 0, 2024-04-25T20:10:51.311 INFO:teuthology.orchestra.run.smithi039.stdout: "expected_num_objects": 0, 2024-04-25T20:10:51.311 INFO:teuthology.orchestra.run.smithi039.stdout: "fast_read": false, 2024-04-25T20:10:51.311 INFO:teuthology.orchestra.run.smithi039.stdout: "options": {}, 2024-04-25T20:10:51.311 INFO:teuthology.orchestra.run.smithi039.stdout: "application_metadata": { 2024-04-25T20:10:51.311 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd": {} 2024-04-25T20:10:51.311 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.311 INFO:teuthology.orchestra.run.smithi039.stdout: "read_balance": { 2024-04-25T20:10:51.311 INFO:teuthology.orchestra.run.smithi039.stdout: "score_acting": 1.7400000095367432, 2024-04-25T20:10:51.311 INFO:teuthology.orchestra.run.smithi039.stdout: "score_stable": 1.7400000095367432, 2024-04-25T20:10:51.311 INFO:teuthology.orchestra.run.smithi039.stdout: "optimal_score": 0.86000001430511475, 2024-04-25T20:10:51.311 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_acting": 1.5, 2024-04-25T20:10:51.312 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_stable": 1.5, 2024-04-25T20:10:51.312 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity_weighted": 0.81888771057128906, 2024-04-25T20:10:51.312 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity": 0.93999999761581421, 2024-04-25T20:10:51.312 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity_weighted": 0.81999999284744263 2024-04-25T20:10:51.312 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.312 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.312 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-04-25T20:10:51.312 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": ".mgr", 2024-04-25T20:10:51.312 INFO:teuthology.orchestra.run.smithi039.stdout: "create_time": "2024-04-25T20:09:15.645688+0000", 2024-04-25T20:10:51.312 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-04-25T20:10:51.312 INFO:teuthology.orchestra.run.smithi039.stdout: "flags_names": "hashpspool", 2024-04-25T20:10:51.312 INFO:teuthology.orchestra.run.smithi039.stdout: "type": 1, 2024-04-25T20:10:51.312 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 2, 2024-04-25T20:10:51.313 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": 1, 2024-04-25T20:10:51.313 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_rule": 0, 2024-04-25T20:10:51.313 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_count": 0, 2024-04-25T20:10:51.313 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_target": 0, 2024-04-25T20:10:51.313 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T20:10:51.313 INFO:teuthology.orchestra.run.smithi039.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T20:10:51.313 INFO:teuthology.orchestra.run.smithi039.stdout: "object_hash": 2, 2024-04-25T20:10:51.313 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscale_mode": "off", 2024-04-25T20:10:51.313 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": 1, 2024-04-25T20:10:51.313 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num": 1, 2024-04-25T20:10:51.313 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_placement_num_target": 1, 2024-04-25T20:10:51.313 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_target": 1, 2024-04-25T20:10:51.313 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_pending": 1, 2024-04-25T20:10:51.313 INFO:teuthology.orchestra.run.smithi039.stdout: "last_pg_merge_meta": { 2024-04-25T20:10:51.313 INFO:teuthology.orchestra.run.smithi039.stdout: "source_pgid": "0.0", 2024-04-25T20:10:51.313 INFO:teuthology.orchestra.run.smithi039.stdout: "ready_epoch": 0, 2024-04-25T20:10:51.314 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_started": 0, 2024-04-25T20:10:51.314 INFO:teuthology.orchestra.run.smithi039.stdout: "last_epoch_clean": 0, 2024-04-25T20:10:51.314 INFO:teuthology.orchestra.run.smithi039.stdout: "source_version": "0'0", 2024-04-25T20:10:51.314 INFO:teuthology.orchestra.run.smithi039.stdout: "target_version": "0'0" 2024-04-25T20:10:51.314 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.314 INFO:teuthology.orchestra.run.smithi039.stdout: "last_change": "17", 2024-04-25T20:10:51.314 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend": "0", 2024-04-25T20:10:51.314 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T20:10:51.314 INFO:teuthology.orchestra.run.smithi039.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T20:10:51.314 INFO:teuthology.orchestra.run.smithi039.stdout: "auid": 0, 2024-04-25T20:10:51.314 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_mode": "selfmanaged", 2024-04-25T20:10:51.314 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_seq": 0, 2024-04-25T20:10:51.314 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_epoch": 0, 2024-04-25T20:10:51.314 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_snaps": [], 2024-04-25T20:10:51.314 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps": "[]", 2024-04-25T20:10:51.315 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_bytes": 0, 2024-04-25T20:10:51.315 INFO:teuthology.orchestra.run.smithi039.stdout: "quota_max_objects": 0, 2024-04-25T20:10:51.315 INFO:teuthology.orchestra.run.smithi039.stdout: "tiers": [], 2024-04-25T20:10:51.315 INFO:teuthology.orchestra.run.smithi039.stdout: "tier_of": -1, 2024-04-25T20:10:51.315 INFO:teuthology.orchestra.run.smithi039.stdout: "read_tier": -1, 2024-04-25T20:10:51.315 INFO:teuthology.orchestra.run.smithi039.stdout: "write_tier": -1, 2024-04-25T20:10:51.315 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_mode": "none", 2024-04-25T20:10:51.315 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_bytes": 0, 2024-04-25T20:10:51.315 INFO:teuthology.orchestra.run.smithi039.stdout: "target_max_objects": 0, 2024-04-25T20:10:51.315 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T20:10:51.315 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T20:10:51.315 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T20:10:51.315 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_flush_age": 0, 2024-04-25T20:10:51.315 INFO:teuthology.orchestra.run.smithi039.stdout: "cache_min_evict_age": 0, 2024-04-25T20:10:51.315 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profile": "", 2024-04-25T20:10:51.315 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_params": { 2024-04-25T20:10:51.316 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "none" 2024-04-25T20:10:51.316 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.316 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_period": 0, 2024-04-25T20:10:51.316 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_count": 0, 2024-04-25T20:10:51.316 INFO:teuthology.orchestra.run.smithi039.stdout: "use_gmt_hitset": true, 2024-04-25T20:10:51.316 INFO:teuthology.orchestra.run.smithi039.stdout: "min_read_recency_for_promote": 0, 2024-04-25T20:10:51.316 INFO:teuthology.orchestra.run.smithi039.stdout: "min_write_recency_for_promote": 0, 2024-04-25T20:10:51.316 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T20:10:51.316 INFO:teuthology.orchestra.run.smithi039.stdout: "hit_set_search_last_n": 0, 2024-04-25T20:10:51.316 INFO:teuthology.orchestra.run.smithi039.stdout: "grade_table": [], 2024-04-25T20:10:51.316 INFO:teuthology.orchestra.run.smithi039.stdout: "stripe_width": 0, 2024-04-25T20:10:51.316 INFO:teuthology.orchestra.run.smithi039.stdout: "expected_num_objects": 0, 2024-04-25T20:10:51.316 INFO:teuthology.orchestra.run.smithi039.stdout: "fast_read": false, 2024-04-25T20:10:51.316 INFO:teuthology.orchestra.run.smithi039.stdout: "options": { 2024-04-25T20:10:51.316 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_max": 32, 2024-04-25T20:10:51.316 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num_min": 1 2024-04-25T20:10:51.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.317 INFO:teuthology.orchestra.run.smithi039.stdout: "application_metadata": { 2024-04-25T20:10:51.317 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": {} 2024-04-25T20:10:51.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.317 INFO:teuthology.orchestra.run.smithi039.stdout: "read_balance": { 2024-04-25T20:10:51.317 INFO:teuthology.orchestra.run.smithi039.stdout: "score_acting": 6.9000000953674316, 2024-04-25T20:10:51.317 INFO:teuthology.orchestra.run.smithi039.stdout: "score_stable": 6.9000000953674316, 2024-04-25T20:10:51.317 INFO:teuthology.orchestra.run.smithi039.stdout: "optimal_score": 0.28999999165534973, 2024-04-25T20:10:51.317 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_acting": 2, 2024-04-25T20:10:51.317 INFO:teuthology.orchestra.run.smithi039.stdout: "raw_score_stable": 2, 2024-04-25T20:10:51.317 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity_weighted": 0.81888771057128906, 2024-04-25T20:10:51.317 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity": 0.93999999761581421, 2024-04-25T20:10:51.317 INFO:teuthology.orchestra.run.smithi039.stdout: "average_primary_affinity_weighted": 0.81999999284744263 2024-04-25T20:10:51.317 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.317 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.318 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-04-25T20:10:51.318 INFO:teuthology.orchestra.run.smithi039.stdout: "osds": [ 2024-04-25T20:10:51.318 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.318 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 0, 2024-04-25T20:10:51.318 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "7a7118ba-4aee-404f-97e3-94c50e2996c4", 2024-04-25T20:10:51.318 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-04-25T20:10:51.318 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-04-25T20:10:51.318 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-04-25T20:10:51.318 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-04-25T20:10:51.318 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-04-25T20:10:51.318 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-04-25T20:10:51.318 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 13, 2024-04-25T20:10:51.318 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 52, 2024-04-25T20:10:51.318 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-04-25T20:10:51.318 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-04-25T20:10:51.319 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-04-25T20:10:51.319 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.319 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.319 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.319 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6816", 2024-04-25T20:10:51.319 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31832 2024-04-25T20:10:51.319 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.319 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.319 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.319 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6818", 2024-04-25T20:10:51.319 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31832 2024-04-25T20:10:51.319 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.319 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.319 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.320 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-04-25T20:10:51.320 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.320 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.320 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.320 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6820", 2024-04-25T20:10:51.320 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31832 2024-04-25T20:10:51.320 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.320 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.320 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.320 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6822", 2024-04-25T20:10:51.320 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31832 2024-04-25T20:10:51.320 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.320 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.320 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.320 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-04-25T20:10:51.321 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.321 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.321 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.321 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6828", 2024-04-25T20:10:51.321 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31832 2024-04-25T20:10:51.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.321 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.321 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.321 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6830", 2024-04-25T20:10:51.321 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31832 2024-04-25T20:10:51.321 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.321 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.321 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-04-25T20:10:51.321 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.322 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.322 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.322 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6824", 2024-04-25T20:10:51.322 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31832 2024-04-25T20:10:51.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.322 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.322 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.322 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6826", 2024-04-25T20:10:51.322 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31832 2024-04-25T20:10:51.322 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.322 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.322 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6818/31832", 2024-04-25T20:10:51.323 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6822/31832", 2024-04-25T20:10:51.323 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6830/31832", 2024-04-25T20:10:51.323 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6826/31832", 2024-04-25T20:10:51.323 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-04-25T20:10:51.323 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-04-25T20:10:51.323 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-04-25T20:10:51.323 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.323 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.323 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.323 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 1, 2024-04-25T20:10:51.323 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "13637dbe-e37f-457f-b68b-8eed1b970617", 2024-04-25T20:10:51.323 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-04-25T20:10:51.323 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-04-25T20:10:51.323 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-04-25T20:10:51.323 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-04-25T20:10:51.323 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-04-25T20:10:51.324 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-04-25T20:10:51.324 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 13, 2024-04-25T20:10:51.324 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 14, 2024-04-25T20:10:51.324 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-04-25T20:10:51.324 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-04-25T20:10:51.324 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-04-25T20:10:51.324 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.324 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.324 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.324 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6808", 2024-04-25T20:10:51.324 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31831 2024-04-25T20:10:51.324 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.324 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.324 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.324 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6809", 2024-04-25T20:10:51.324 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31831 2024-04-25T20:10:51.325 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.325 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.325 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.325 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-04-25T20:10:51.325 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.325 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.325 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.325 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6810", 2024-04-25T20:10:51.325 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31831 2024-04-25T20:10:51.325 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.325 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.325 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.325 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6811", 2024-04-25T20:10:51.325 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31831 2024-04-25T20:10:51.325 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.325 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.326 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-04-25T20:10:51.326 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.326 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.326 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.326 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6814", 2024-04-25T20:10:51.326 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31831 2024-04-25T20:10:51.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.326 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.326 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.326 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6815", 2024-04-25T20:10:51.326 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31831 2024-04-25T20:10:51.326 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.326 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.327 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-04-25T20:10:51.327 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.327 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.327 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.327 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6812", 2024-04-25T20:10:51.327 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31831 2024-04-25T20:10:51.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.327 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.327 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.327 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6813", 2024-04-25T20:10:51.327 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31831 2024-04-25T20:10:51.327 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.327 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.327 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6809/31831", 2024-04-25T20:10:51.328 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6811/31831", 2024-04-25T20:10:51.328 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6815/31831", 2024-04-25T20:10:51.328 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6813/31831", 2024-04-25T20:10:51.328 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-04-25T20:10:51.328 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-04-25T20:10:51.328 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-04-25T20:10:51.328 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.328 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.328 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.328 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 2, 2024-04-25T20:10:51.328 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "29e20ce2-4776-455d-8cf4-d539e738a6a7", 2024-04-25T20:10:51.328 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-04-25T20:10:51.328 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-04-25T20:10:51.328 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 0.7264251708984375, 2024-04-25T20:10:51.328 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-04-25T20:10:51.329 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-04-25T20:10:51.329 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-04-25T20:10:51.329 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 13, 2024-04-25T20:10:51.329 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 0, 2024-04-25T20:10:51.329 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-04-25T20:10:51.329 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-04-25T20:10:51.329 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-04-25T20:10:51.329 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.329 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.329 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.329 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6800", 2024-04-25T20:10:51.329 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31829 2024-04-25T20:10:51.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.329 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.329 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.330 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6801", 2024-04-25T20:10:51.330 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31829 2024-04-25T20:10:51.330 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.330 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.330 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-04-25T20:10:51.330 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.330 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.330 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.330 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6802", 2024-04-25T20:10:51.330 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31829 2024-04-25T20:10:51.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.330 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.330 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.331 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6803", 2024-04-25T20:10:51.331 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31829 2024-04-25T20:10:51.331 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.331 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.331 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-04-25T20:10:51.331 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.331 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.331 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6806", 2024-04-25T20:10:51.331 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31829 2024-04-25T20:10:51.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.331 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.331 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6807", 2024-04-25T20:10:51.331 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31829 2024-04-25T20:10:51.332 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.332 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.332 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-04-25T20:10:51.332 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.332 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.332 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6804", 2024-04-25T20:10:51.332 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31829 2024-04-25T20:10:51.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.332 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.332 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6805", 2024-04-25T20:10:51.332 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31829 2024-04-25T20:10:51.332 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.332 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.333 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6801/31829", 2024-04-25T20:10:51.333 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6803/31829", 2024-04-25T20:10:51.333 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6807/31829", 2024-04-25T20:10:51.333 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6805/31829", 2024-04-25T20:10:51.333 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-04-25T20:10:51.333 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-04-25T20:10:51.333 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-04-25T20:10:51.333 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.333 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.333 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 3, 2024-04-25T20:10:51.333 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "a8b34e02-0898-4303-9fd5-62cbba9fec5c", 2024-04-25T20:10:51.333 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-04-25T20:10:51.333 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 0, 2024-04-25T20:10:51.333 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 0, 2024-04-25T20:10:51.334 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-04-25T20:10:51.334 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-04-25T20:10:51.334 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-04-25T20:10:51.334 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 13, 2024-04-25T20:10:51.334 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 14, 2024-04-25T20:10:51.334 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-04-25T20:10:51.334 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-04-25T20:10:51.334 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-04-25T20:10:51.334 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.334 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.334 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.334 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6817", 2024-04-25T20:10:51.334 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31830 2024-04-25T20:10:51.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.334 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.334 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.335 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6819", 2024-04-25T20:10:51.335 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31830 2024-04-25T20:10:51.335 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.335 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.335 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-04-25T20:10:51.335 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.335 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.335 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6821", 2024-04-25T20:10:51.335 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31830 2024-04-25T20:10:51.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.335 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.335 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6823", 2024-04-25T20:10:51.335 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31830 2024-04-25T20:10:51.336 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.336 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.336 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-04-25T20:10:51.336 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.336 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.336 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.336 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6829", 2024-04-25T20:10:51.336 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31830 2024-04-25T20:10:51.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.336 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.336 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.336 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6831", 2024-04-25T20:10:51.336 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31830 2024-04-25T20:10:51.336 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.336 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.337 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-04-25T20:10:51.337 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.337 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.337 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.337 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6825", 2024-04-25T20:10:51.337 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31830 2024-04-25T20:10:51.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.337 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.337 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.337 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6827", 2024-04-25T20:10:51.337 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31830 2024-04-25T20:10:51.337 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.337 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.338 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.39:6819/31830", 2024-04-25T20:10:51.338 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.39:6823/31830", 2024-04-25T20:10:51.338 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.39:6831/31830", 2024-04-25T20:10:51.338 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.39:6827/31830", 2024-04-25T20:10:51.338 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-04-25T20:10:51.338 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-04-25T20:10:51.338 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-04-25T20:10:51.338 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.338 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.338 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 4, 2024-04-25T20:10:51.338 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "e92b5c67-6908-4fd4-9940-5cb98b5ee847", 2024-04-25T20:10:51.338 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-04-25T20:10:51.338 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-04-25T20:10:51.339 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-04-25T20:10:51.339 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 0.5724334716796875, 2024-04-25T20:10:51.339 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-04-25T20:10:51.339 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-04-25T20:10:51.339 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 13, 2024-04-25T20:10:51.339 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 34, 2024-04-25T20:10:51.339 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-04-25T20:10:51.339 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-04-25T20:10:51.339 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-04-25T20:10:51.339 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.339 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.339 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.339 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6800", 2024-04-25T20:10:51.339 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31724 2024-04-25T20:10:51.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.340 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.340 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.340 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6801", 2024-04-25T20:10:51.340 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31724 2024-04-25T20:10:51.340 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.340 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.340 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-04-25T20:10:51.340 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.340 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.340 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.340 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6802", 2024-04-25T20:10:51.340 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31724 2024-04-25T20:10:51.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.341 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.341 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.341 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6803", 2024-04-25T20:10:51.341 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31724 2024-04-25T20:10:51.341 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.341 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.341 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-04-25T20:10:51.341 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.341 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.341 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.341 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6806", 2024-04-25T20:10:51.341 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31724 2024-04-25T20:10:51.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.342 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.342 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6807", 2024-04-25T20:10:51.342 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31724 2024-04-25T20:10:51.342 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.342 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.342 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-04-25T20:10:51.342 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.342 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.342 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6804", 2024-04-25T20:10:51.342 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31724 2024-04-25T20:10:51.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.343 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.343 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.343 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6805", 2024-04-25T20:10:51.343 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31724 2024-04-25T20:10:51.343 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.343 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.343 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.135:6801/31724", 2024-04-25T20:10:51.343 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.135:6803/31724", 2024-04-25T20:10:51.343 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.135:6807/31724", 2024-04-25T20:10:51.343 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.135:6805/31724", 2024-04-25T20:10:51.343 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-04-25T20:10:51.343 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-04-25T20:10:51.344 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-04-25T20:10:51.344 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.344 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.344 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 5, 2024-04-25T20:10:51.344 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "d74d8fef-0211-413b-bff2-f97776fc1faa", 2024-04-25T20:10:51.344 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-04-25T20:10:51.344 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-04-25T20:10:51.344 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-04-25T20:10:51.344 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 0, 2024-04-25T20:10:51.344 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-04-25T20:10:51.344 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-04-25T20:10:51.344 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 13, 2024-04-25T20:10:51.344 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 14, 2024-04-25T20:10:51.344 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-04-25T20:10:51.345 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-04-25T20:10:51.345 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-04-25T20:10:51.345 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.345 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.345 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.345 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6820", 2024-04-25T20:10:51.345 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31734 2024-04-25T20:10:51.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.345 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.345 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.345 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6822", 2024-04-25T20:10:51.345 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31734 2024-04-25T20:10:51.345 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.345 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.346 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-04-25T20:10:51.346 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.346 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.346 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.346 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6824", 2024-04-25T20:10:51.346 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31734 2024-04-25T20:10:51.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.346 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.346 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.346 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6826", 2024-04-25T20:10:51.346 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31734 2024-04-25T20:10:51.346 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.346 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.347 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-04-25T20:10:51.347 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.347 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.347 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6830", 2024-04-25T20:10:51.347 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31734 2024-04-25T20:10:51.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.347 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.347 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.347 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6831", 2024-04-25T20:10:51.347 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31734 2024-04-25T20:10:51.347 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.347 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.347 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-04-25T20:10:51.348 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.348 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.348 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6828", 2024-04-25T20:10:51.348 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31734 2024-04-25T20:10:51.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.348 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.348 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6829", 2024-04-25T20:10:51.348 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31734 2024-04-25T20:10:51.348 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.348 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.348 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.135:6822/31734", 2024-04-25T20:10:51.348 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.135:6826/31734", 2024-04-25T20:10:51.349 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.135:6831/31734", 2024-04-25T20:10:51.349 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.135:6829/31734", 2024-04-25T20:10:51.349 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-04-25T20:10:51.349 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-04-25T20:10:51.349 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-04-25T20:10:51.349 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.349 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.349 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 6, 2024-04-25T20:10:51.349 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "7dc84c6e-76d8-4f22-8b17-51f62765762d", 2024-04-25T20:10:51.349 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-04-25T20:10:51.349 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-04-25T20:10:51.349 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 1, 2024-04-25T20:10:51.349 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 0.978668212890625, 2024-04-25T20:10:51.349 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-04-25T20:10:51.350 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-04-25T20:10:51.350 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 13, 2024-04-25T20:10:51.350 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 55, 2024-04-25T20:10:51.350 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-04-25T20:10:51.350 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-04-25T20:10:51.350 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-04-25T20:10:51.350 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.350 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.350 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.350 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6808", 2024-04-25T20:10:51.350 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31725 2024-04-25T20:10:51.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.350 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.350 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.351 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6809", 2024-04-25T20:10:51.351 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31725 2024-04-25T20:10:51.351 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.351 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.351 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-04-25T20:10:51.351 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.351 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.351 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.351 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6810", 2024-04-25T20:10:51.351 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31725 2024-04-25T20:10:51.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.351 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.351 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.352 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6811", 2024-04-25T20:10:51.352 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31725 2024-04-25T20:10:51.352 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.352 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.352 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-04-25T20:10:51.352 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.352 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.352 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6814", 2024-04-25T20:10:51.352 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31725 2024-04-25T20:10:51.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.352 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.352 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.352 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6815", 2024-04-25T20:10:51.353 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31725 2024-04-25T20:10:51.353 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.353 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.353 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-04-25T20:10:51.353 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.353 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.353 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6812", 2024-04-25T20:10:51.353 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31725 2024-04-25T20:10:51.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.353 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.353 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.353 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6813", 2024-04-25T20:10:51.354 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31725 2024-04-25T20:10:51.354 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.354 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.354 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.135:6809/31725", 2024-04-25T20:10:51.354 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.135:6811/31725", 2024-04-25T20:10:51.354 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.135:6815/31725", 2024-04-25T20:10:51.354 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.135:6813/31725", 2024-04-25T20:10:51.354 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-04-25T20:10:51.354 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-04-25T20:10:51.354 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-04-25T20:10:51.354 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.355 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.355 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 7, 2024-04-25T20:10:51.355 INFO:teuthology.orchestra.run.smithi039.stdout: "uuid": "7adb3187-aed3-48af-868c-7230e9bfc85e", 2024-04-25T20:10:51.355 INFO:teuthology.orchestra.run.smithi039.stdout: "up": 1, 2024-04-25T20:10:51.355 INFO:teuthology.orchestra.run.smithi039.stdout: "in": 1, 2024-04-25T20:10:51.355 INFO:teuthology.orchestra.run.smithi039.stdout: "weight": 0.9459075927734375, 2024-04-25T20:10:51.355 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_affinity": 1, 2024-04-25T20:10:51.355 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_begin": 0, 2024-04-25T20:10:51.355 INFO:teuthology.orchestra.run.smithi039.stdout: "last_clean_end": 0, 2024-04-25T20:10:51.355 INFO:teuthology.orchestra.run.smithi039.stdout: "up_from": 13, 2024-04-25T20:10:51.355 INFO:teuthology.orchestra.run.smithi039.stdout: "up_thru": 25, 2024-04-25T20:10:51.355 INFO:teuthology.orchestra.run.smithi039.stdout: "down_at": 0, 2024-04-25T20:10:51.355 INFO:teuthology.orchestra.run.smithi039.stdout: "lost_at": 0, 2024-04-25T20:10:51.355 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addrs": { 2024-04-25T20:10:51.355 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.356 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.356 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.356 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6816", 2024-04-25T20:10:51.356 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31737 2024-04-25T20:10:51.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.356 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.356 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.356 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6817", 2024-04-25T20:10:51.356 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31737 2024-04-25T20:10:51.356 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.356 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.356 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addrs": { 2024-04-25T20:10:51.356 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.356 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.357 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.357 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6818", 2024-04-25T20:10:51.357 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31737 2024-04-25T20:10:51.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.357 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.357 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.357 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6819", 2024-04-25T20:10:51.357 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31737 2024-04-25T20:10:51.357 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.357 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.357 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addrs": { 2024-04-25T20:10:51.357 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.357 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.358 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.358 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6825", 2024-04-25T20:10:51.358 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31737 2024-04-25T20:10:51.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.358 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.358 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.358 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6827", 2024-04-25T20:10:51.358 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31737 2024-04-25T20:10:51.358 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.358 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.358 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addrs": { 2024-04-25T20:10:51.358 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-04-25T20:10:51.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.359 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-04-25T20:10:51.359 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6821", 2024-04-25T20:10:51.359 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31737 2024-04-25T20:10:51.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.359 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-04-25T20:10:51.359 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.135:6823", 2024-04-25T20:10:51.359 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 31737 2024-04-25T20:10:51.359 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.359 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.359 INFO:teuthology.orchestra.run.smithi039.stdout: "public_addr": "172.21.15.135:6817/31737", 2024-04-25T20:10:51.359 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_addr": "172.21.15.135:6819/31737", 2024-04-25T20:10:51.360 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_back_addr": "172.21.15.135:6827/31737", 2024-04-25T20:10:51.360 INFO:teuthology.orchestra.run.smithi039.stdout: "heartbeat_front_addr": "172.21.15.135:6823/31737", 2024-04-25T20:10:51.360 INFO:teuthology.orchestra.run.smithi039.stdout: "state": [ 2024-04-25T20:10:51.360 INFO:teuthology.orchestra.run.smithi039.stdout: "exists", 2024-04-25T20:10:51.360 INFO:teuthology.orchestra.run.smithi039.stdout: "up" 2024-04-25T20:10:51.360 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.360 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.360 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-04-25T20:10:51.360 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_xinfo": [ 2024-04-25T20:10:51.360 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.360 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 0, 2024-04-25T20:10:51.360 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-04-25T20:10:51.360 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-04-25T20:10:51.360 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-04-25T20:10:51.360 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540138322906710015, 2024-04-25T20:10:51.361 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-04-25T20:10:51.361 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-04-25T20:09:10.683465+0000", 2024-04-25T20:10:51.361 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-04-25T20:10:51.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.361 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.361 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 1, 2024-04-25T20:10:51.361 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-04-25T20:10:51.361 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-04-25T20:10:51.361 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-04-25T20:10:51.361 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540138322906710015, 2024-04-25T20:10:51.361 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-04-25T20:10:51.361 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-04-25T20:09:10.722423+0000", 2024-04-25T20:10:51.361 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-04-25T20:10:51.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.361 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.361 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 2, 2024-04-25T20:10:51.362 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-04-25T20:10:51.362 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-04-25T20:10:51.362 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-04-25T20:10:51.362 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540138322906710015, 2024-04-25T20:10:51.362 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-04-25T20:10:51.362 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-04-25T20:09:10.712691+0000", 2024-04-25T20:10:51.362 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-04-25T20:10:51.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.362 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 3, 2024-04-25T20:10:51.362 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-04-25T20:10:51.362 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-04-25T20:10:51.362 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-04-25T20:10:51.362 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540138322906710015, 2024-04-25T20:10:51.363 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 65536, 2024-04-25T20:10:51.363 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-04-25T20:09:10.673892+0000", 2024-04-25T20:10:51.363 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-04-25T20:10:51.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.363 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 4, 2024-04-25T20:10:51.363 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-04-25T20:10:51.363 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-04-25T20:10:51.363 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-04-25T20:10:51.363 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540138322906710015, 2024-04-25T20:10:51.363 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-04-25T20:10:51.363 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-04-25T20:09:10.768469+0000", 2024-04-25T20:10:51.363 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-04-25T20:10:51.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.363 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.364 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 5, 2024-04-25T20:10:51.364 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-04-25T20:10:51.364 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-04-25T20:10:51.364 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-04-25T20:10:51.364 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540138322906710015, 2024-04-25T20:10:51.364 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-04-25T20:10:51.364 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-04-25T20:09:10.783897+0000", 2024-04-25T20:10:51.364 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-04-25T20:10:51.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.364 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.364 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 6, 2024-04-25T20:10:51.364 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-04-25T20:10:51.364 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-04-25T20:10:51.364 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-04-25T20:10:51.364 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540138322906710015, 2024-04-25T20:10:51.365 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-04-25T20:10:51.365 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-04-25T20:09:10.686833+0000", 2024-04-25T20:10:51.365 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-04-25T20:10:51.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.365 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.365 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": 7, 2024-04-25T20:10:51.365 INFO:teuthology.orchestra.run.smithi039.stdout: "down_stamp": "0.000000", 2024-04-25T20:10:51.365 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_probability": 0, 2024-04-25T20:10:51.365 INFO:teuthology.orchestra.run.smithi039.stdout: "laggy_interval": 0, 2024-04-25T20:10:51.365 INFO:teuthology.orchestra.run.smithi039.stdout: "features": 4540138322906710015, 2024-04-25T20:10:51.365 INFO:teuthology.orchestra.run.smithi039.stdout: "old_weight": 0, 2024-04-25T20:10:51.365 INFO:teuthology.orchestra.run.smithi039.stdout: "last_purged_snaps_scrub": "2024-04-25T20:09:10.739675+0000", 2024-04-25T20:10:51.365 INFO:teuthology.orchestra.run.smithi039.stdout: "dead_epoch": 0 2024-04-25T20:10:51.365 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.365 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-04-25T20:10:51.366 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_upmap": [], 2024-04-25T20:10:51.366 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_upmap_items": [ 2024-04-25T20:10:51.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.366 INFO:teuthology.orchestra.run.smithi039.stdout: "pgid": "1.7", 2024-04-25T20:10:51.366 INFO:teuthology.orchestra.run.smithi039.stdout: "mappings": [ 2024-04-25T20:10:51.366 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.366 INFO:teuthology.orchestra.run.smithi039.stdout: "from": 7, 2024-04-25T20:10:51.366 INFO:teuthology.orchestra.run.smithi039.stdout: "to": 1 2024-04-25T20:10:51.366 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.366 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.366 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.366 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-04-25T20:10:51.366 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_upmap_primaries": [], 2024-04-25T20:10:51.366 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_temp": [], 2024-04-25T20:10:51.366 INFO:teuthology.orchestra.run.smithi039.stdout: "primary_temp": [], 2024-04-25T20:10:51.367 INFO:teuthology.orchestra.run.smithi039.stdout: "blocklist": {}, 2024-04-25T20:10:51.367 INFO:teuthology.orchestra.run.smithi039.stdout: "range_blocklist": {}, 2024-04-25T20:10:51.367 INFO:teuthology.orchestra.run.smithi039.stdout: "erasure_code_profiles": { 2024-04-25T20:10:51.367 INFO:teuthology.orchestra.run.smithi039.stdout: "default": { 2024-04-25T20:10:51.367 INFO:teuthology.orchestra.run.smithi039.stdout: "crush-failure-domain": "osd", 2024-04-25T20:10:51.367 INFO:teuthology.orchestra.run.smithi039.stdout: "k": "2", 2024-04-25T20:10:51.367 INFO:teuthology.orchestra.run.smithi039.stdout: "m": "1", 2024-04-25T20:10:51.367 INFO:teuthology.orchestra.run.smithi039.stdout: "plugin": "jerasure", 2024-04-25T20:10:51.367 INFO:teuthology.orchestra.run.smithi039.stdout: "technique": "reed_sol_van" 2024-04-25T20:10:51.367 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.367 INFO:teuthology.orchestra.run.smithi039.stdout: "removed_snaps_queue": [ 2024-04-25T20:10:51.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.367 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 1, 2024-04-25T20:10:51.367 INFO:teuthology.orchestra.run.smithi039.stdout: "snaps": [ 2024-04-25T20:10:51.367 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.368 INFO:teuthology.orchestra.run.smithi039.stdout: "begin": 2, 2024-04-25T20:10:51.368 INFO:teuthology.orchestra.run.smithi039.stdout: "length": 1 2024-04-25T20:10:51.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:51.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.368 INFO:teuthology.orchestra.run.smithi039.stdout: "begin": 4, 2024-04-25T20:10:51.368 INFO:teuthology.orchestra.run.smithi039.stdout: "length": 1 2024-04-25T20:10:51.368 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.368 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.368 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.368 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-04-25T20:10:51.368 INFO:teuthology.orchestra.run.smithi039.stdout: "new_removed_snaps": [ 2024-04-25T20:10:51.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.368 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 1, 2024-04-25T20:10:51.368 INFO:teuthology.orchestra.run.smithi039.stdout: "snaps": [ 2024-04-25T20:10:51.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:51.368 INFO:teuthology.orchestra.run.smithi039.stdout: "begin": 4, 2024-04-25T20:10:51.369 INFO:teuthology.orchestra.run.smithi039.stdout: "length": 1 2024-04-25T20:10:51.369 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.369 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:51.369 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.369 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-04-25T20:10:51.369 INFO:teuthology.orchestra.run.smithi039.stdout: "new_purged_snaps": [], 2024-04-25T20:10:51.369 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_node_flags": {}, 2024-04-25T20:10:51.369 INFO:teuthology.orchestra.run.smithi039.stdout: "device_class_flags": {}, 2024-04-25T20:10:51.369 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_mode": { 2024-04-25T20:10:51.369 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_mode_enabled": false, 2024-04-25T20:10:51.369 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_bucket_count": 0, 2024-04-25T20:10:51.369 INFO:teuthology.orchestra.run.smithi039.stdout: "degraded_stretch_mode": 0, 2024-04-25T20:10:51.369 INFO:teuthology.orchestra.run.smithi039.stdout: "recovering_stretch_mode": 0, 2024-04-25T20:10:51.369 INFO:teuthology.orchestra.run.smithi039.stdout: "stretch_mode_bucket": 0 2024-04-25T20:10:51.369 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:51.370 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:51.371 INFO:tasks.workunit.client.0.smithi039.stderr: Removing image: 40% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 48% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 62% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 73% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 87% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 98% complete... Removing image: 100% complete...done. 2024-04-25T20:10:51.372 INFO:tasks.thrashosds.thrasher:j is {'epoch': 61, 'fsid': '975ceafd-555d-4f71-81e4-1f70364da00f', 'created': '2024-04-25T20:09:03.377320+0000', 'modified': '2024-04-25T20:10:49.982724+0000', 'last_up_change': '2024-04-25T20:09:13.740341+0000', 'last_in_change': '2024-04-25T20:10:45.945659+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-04-25T20:09:14.202057+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8600000143051147, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.8188877105712891, 'average_primary_affinity': 0.9399999976158142, 'average_primary_affinity_weighted': 0.8199999928474426}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-04-25T20:09:15.645688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8188877105712891, 'average_primary_affinity': 0.9399999976158142, 'average_primary_affinity_weighted': 0.8199999928474426}}], 'osds': [{'osd': 0, 'uuid': '7a7118ba-4aee-404f-97e3-94c50e2996c4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 52, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6816', 'nonce': 31832}, {'type': 'v1', 'addr': '172.21.15.39:6818', 'nonce': 31832}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6820', 'nonce': 31832}, {'type': 'v1', 'addr': '172.21.15.39:6822', 'nonce': 31832}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6828', 'nonce': 31832}, {'type': 'v1', 'addr': '172.21.15.39:6830', 'nonce': 31832}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6824', 'nonce': 31832}, {'type': 'v1', 'addr': '172.21.15.39:6826', 'nonce': 31832}]}, 'public_addr': '172.21.15.39:6818/31832', 'cluster_addr': '172.21.15.39:6822/31832', 'heartbeat_back_addr': '172.21.15.39:6830/31832', 'heartbeat_front_addr': '172.21.15.39:6826/31832', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '13637dbe-e37f-457f-b68b-8eed1b970617', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 14, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6808', 'nonce': 31831}, {'type': 'v1', 'addr': '172.21.15.39:6809', 'nonce': 31831}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6810', 'nonce': 31831}, {'type': 'v1', 'addr': '172.21.15.39:6811', 'nonce': 31831}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6814', 'nonce': 31831}, {'type': 'v1', 'addr': '172.21.15.39:6815', 'nonce': 31831}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6812', 'nonce': 31831}, {'type': 'v1', 'addr': '172.21.15.39:6813', 'nonce': 31831}]}, 'public_addr': '172.21.15.39:6809/31831', 'cluster_addr': '172.21.15.39:6811/31831', 'heartbeat_back_addr': '172.21.15.39:6815/31831', 'heartbeat_front_addr': '172.21.15.39:6813/31831', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '29e20ce2-4776-455d-8cf4-d539e738a6a7', 'up': 1, 'in': 1, 'weight': 0.7264251708984375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6800', 'nonce': 31829}, {'type': 'v1', 'addr': '172.21.15.39:6801', 'nonce': 31829}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6802', 'nonce': 31829}, {'type': 'v1', 'addr': '172.21.15.39:6803', 'nonce': 31829}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6806', 'nonce': 31829}, {'type': 'v1', 'addr': '172.21.15.39:6807', 'nonce': 31829}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6804', 'nonce': 31829}, {'type': 'v1', 'addr': '172.21.15.39:6805', 'nonce': 31829}]}, 'public_addr': '172.21.15.39:6801/31829', 'cluster_addr': '172.21.15.39:6803/31829', 'heartbeat_back_addr': '172.21.15.39:6807/31829', 'heartbeat_front_addr': '172.21.15.39:6805/31829', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'a8b34e02-0898-4303-9fd5-62cbba9fec5c', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 14, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6817', 'nonce': 31830}, {'type': 'v1', 'addr': '172.21.15.39:6819', 'nonce': 31830}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6821', 'nonce': 31830}, {'type': 'v1', 'addr': '172.21.15.39:6823', 'nonce': 31830}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6829', 'nonce': 31830}, {'type': 'v1', 'addr': '172.21.15.39:6831', 'nonce': 31830}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6825', 'nonce': 31830}, {'type': 'v1', 'addr': '172.21.15.39:6827', 'nonce': 31830}]}, 'public_addr': '172.21.15.39:6819/31830', 'cluster_addr': '172.21.15.39:6823/31830', 'heartbeat_back_addr': '172.21.15.39:6831/31830', 'heartbeat_front_addr': '172.21.15.39:6827/31830', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'e92b5c67-6908-4fd4-9940-5cb98b5ee847', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.5724334716796875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6800', 'nonce': 31724}, {'type': 'v1', 'addr': '172.21.15.135:6801', 'nonce': 31724}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6802', 'nonce': 31724}, {'type': 'v1', 'addr': '172.21.15.135:6803', 'nonce': 31724}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6806', 'nonce': 31724}, {'type': 'v1', 'addr': '172.21.15.135:6807', 'nonce': 31724}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6804', 'nonce': 31724}, {'type': 'v1', 'addr': '172.21.15.135:6805', 'nonce': 31724}]}, 'public_addr': '172.21.15.135:6801/31724', 'cluster_addr': '172.21.15.135:6803/31724', 'heartbeat_back_addr': '172.21.15.135:6807/31724', 'heartbeat_front_addr': '172.21.15.135:6805/31724', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'd74d8fef-0211-413b-bff2-f97776fc1faa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 14, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6820', 'nonce': 31734}, {'type': 'v1', 'addr': '172.21.15.135:6822', 'nonce': 31734}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6824', 'nonce': 31734}, {'type': 'v1', 'addr': '172.21.15.135:6826', 'nonce': 31734}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6830', 'nonce': 31734}, {'type': 'v1', 'addr': '172.21.15.135:6831', 'nonce': 31734}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6828', 'nonce': 31734}, {'type': 'v1', 'addr': '172.21.15.135:6829', 'nonce': 31734}]}, 'public_addr': '172.21.15.135:6822/31734', 'cluster_addr': '172.21.15.135:6826/31734', 'heartbeat_back_addr': '172.21.15.135:6831/31734', 'heartbeat_front_addr': '172.21.15.135:6829/31734', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '7dc84c6e-76d8-4f22-8b17-51f62765762d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.978668212890625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 55, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6808', 'nonce': 31725}, {'type': 'v1', 'addr': '172.21.15.135:6809', 'nonce': 31725}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6810', 'nonce': 31725}, {'type': 'v1', 'addr': '172.21.15.135:6811', 'nonce': 31725}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6814', 'nonce': 31725}, {'type': 'v1', 'addr': '172.21.15.135:6815', 'nonce': 31725}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6812', 'nonce': 31725}, {'type': 'v1', 'addr': '172.21.15.135:6813', 'nonce': 31725}]}, 'public_addr': '172.21.15.135:6809/31725', 'cluster_addr': '172.21.15.135:6811/31725', 'heartbeat_back_addr': '172.21.15.135:6815/31725', 'heartbeat_front_addr': '172.21.15.135:6813/31725', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '7adb3187-aed3-48af-868c-7230e9bfc85e', 'up': 1, 'in': 1, 'weight': 0.9459075927734375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6816', 'nonce': 31737}, {'type': 'v1', 'addr': '172.21.15.135:6817', 'nonce': 31737}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6818', 'nonce': 31737}, {'type': 'v1', 'addr': '172.21.15.135:6819', 'nonce': 31737}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6825', 'nonce': 31737}, {'type': 'v1', 'addr': '172.21.15.135:6827', 'nonce': 31737}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6821', 'nonce': 31737}, {'type': 'v1', 'addr': '172.21.15.135:6823', 'nonce': 31737}]}, 'public_addr': '172.21.15.135:6817/31737', 'cluster_addr': '172.21.15.135:6819/31737', 'heartbeat_back_addr': '172.21.15.135:6827/31737', 'heartbeat_front_addr': '172.21.15.135:6823/31737', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T20:09:10.683465+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T20:09:10.722423+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T20:09:10.712691+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-04-25T20:09:10.673892+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T20:09:10.768469+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T20:09:10.783897+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T20:09:10.686833+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-25T20:09:10.739675+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '1.7', 'mappings': [{'from': 7, 'to': 1}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 1, 'snaps': [{'begin': 2, 'length': 1}, {'begin': 4, 'length': 1}]}], 'new_removed_snaps': [{'pool': 1, 'snaps': [{'begin': 4, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-04-25T20:10:51.372 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T20:10:51.373 INFO:tasks.workunit.client.0.smithi039.stderr:+ set +x 2024-04-25T20:10:51.373 INFO:tasks.workunit.client.0.smithi039.stderr:+ test_rbd_clone 2024-04-25T20:10:51.373 INFO:tasks.workunit.client.0.smithi039.stderr:+ local parent=testrbdclonep47106 2024-04-25T20:10:51.373 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd create --image-feature layering --size 256 testrbdclonep47106 2024-04-25T20:10:51.373 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd snap create testrbdclonep47106@snap 2024-04-25T20:10:51.374 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:51.371+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:51.471 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:51.471+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:51.472 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:51.472 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:51.481 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T20:10:51.572 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:51.572+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:51.581 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:51.582 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:51.590 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T20:10:51.673 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:51.672+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:51.691 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:51.691 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:51.698 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:10:51.699 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-04-25T20:10:51.700 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-25T20:10:51.711 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-04-25T20:10:51.617033+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":714343,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":853,"num_read_kb":957,"num_write":295,"num_write_kb":2671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":256,"ondisk_log_size":256,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":6,"kb":660602880,"kb_used":206668,"kb_used_data":5240,"kb_used_omap":10,"kb_used_meta":201333,"kb_avail":660396212,"statfs":{"total":676457349120,"available":676245721088,"internally_reserved":0,"allocated":5365760,"data_stored":3513504,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":11149,"internal_metadata":206165107},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":123975,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":737,"num_read_kb":857,"num_write":136,"num_write_kb":255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.665205"},"pg_stats":[{"pgid":"1.7","version":"59'13","reported_seq":89,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T20:10:49.306490+0000","last_change":"2024-04-25T20:10:46.993500+0000","last_active":"2024-04-25T20:10:49.306490+0000","last_peered":"2024-04-25T20:10:49.306490+0000","last_clean":"2024-04-25T20:10:49.306490+0000","last_became_active":"2024-04-25T20:10:40.927130+0000","last_became_peered":"2024-04-25T20:10:40.927130+0000","last_unstale":"2024-04-25T20:10:49.306490+0000","last_undegraded":"2024-04-25T20:10:49.306490+0000","last_fullsized":"2024-04-25T20:10:49.306490+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:21:16.297993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000264057,"stat_sum":{"num_bytes":53479,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":66,"num_write":13,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"60'34","reported_seq":159,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T20:10:49.611899+0000","last_change":"2024-04-25T20:10:46.993657+0000","last_active":"2024-04-25T20:10:49.611899+0000","last_peered":"2024-04-25T20:10:49.611899+0000","last_clean":"2024-04-25T20:10:49.611899+0000","last_became_active":"2024-04-25T20:09:16.140616+0000","last_became_peered":"2024-04-25T20:09:16.140616+0000","last_unstale":"2024-04-25T20:10:49.611899+0000","last_undegraded":"2024-04-25T20:10:49.611899+0000","last_fullsized":"2024-04-25T20:10:49.611899+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:31.562175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040654999999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":130,"num_read_kb":128,"num_write":45,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"60'23","reported_seq":226,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T20:10:49.565109+0000","last_change":"2024-04-25T20:10:46.972590+0000","last_active":"2024-04-25T20:10:49.565109+0000","last_peered":"2024-04-25T20:10:49.565109+0000","last_clean":"2024-04-25T20:10:49.565109+0000","last_became_active":"2024-04-25T20:10:03.784247+0000","last_became_peered":"2024-04-25T20:10:03.784247+0000","last_unstale":"2024-04-25T20:10:49.565109+0000","last_undegraded":"2024-04-25T20:10:49.565109+0000","last_fullsized":"2024-04-25T20:10:49.565109+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:35.219740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000320501,"stat_sum":{"num_bytes":16,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":302,"num_read_kb":259,"num_write":14,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"60'27","reported_seq":85,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T20:10:49.543156+0000","last_change":"2024-04-25T20:10:46.984333+0000","last_active":"2024-04-25T20:10:49.543156+0000","last_peered":"2024-04-25T20:10:49.543156+0000","last_clean":"2024-04-25T20:10:49.543156+0000","last_became_active":"2024-04-25T20:09:41.938314+0000","last_became_peered":"2024-04-25T20:09:41.938314+0000","last_unstale":"2024-04-25T20:10:49.543156+0000","last_undegraded":"2024-04-25T20:10:49.543156+0000","last_fullsized":"2024-04-25T20:10:49.543156+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:01:12.669404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091904200000000001,"stat_sum":{"num_bytes":57849,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":27,"num_read_kb":133,"num_write":25,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"17'117","reported_seq":200,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T20:10:46.983365+0000","last_change":"2024-04-25T20:09:17.121148+0000","last_active":"2024-04-25T20:10:46.983365+0000","last_peered":"2024-04-25T20:10:46.983365+0000","last_clean":"2024-04-25T20:10:46.983365+0000","last_became_active":"2024-04-25T20:09:17.120312+0000","last_became_peered":"2024-04-25T20:09:17.120312+0000","last_unstale":"2024-04-25T20:10:46.983365+0000","last_undegraded":"2024-04-25T20:10:46.983365+0000","last_fullsized":"2024-04-25T20:10:46.983365+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_clean_scrub_stamp":"2024-04-25T20:09:15.745353+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:48.383841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":57,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T20:10:46.977495+0000","last_change":"2024-04-25T20:10:46.977495+0000","last_active":"2024-04-25T20:10:46.977495+0000","last_peered":"2024-04-25T20:10:46.977495+0000","last_clean":"2024-04-25T20:10:46.977495+0000","last_became_active":"2024-04-25T20:09:15.772551+0000","last_became_peered":"2024-04-25T20:09:15.772551+0000","last_unstale":"2024-04-25T20:10:46.977495+0000","last_undegraded":"2024-04-25T20:10:46.977495+0000","last_fullsized":"2024-04-25T20:10:46.977495+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:55.101025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031613300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"60'6","reported_seq":50,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T20:10:49.533145+0000","last_change":"2024-04-25T20:10:46.980672+0000","last_active":"2024-04-25T20:10:49.533145+0000","last_peered":"2024-04-25T20:10:49.533145+0000","last_clean":"2024-04-25T20:10:49.533145+0000","last_became_active":"2024-04-25T20:10:35.034287+0000","last_became_peered":"2024-04-25T20:10:35.034287+0000","last_unstale":"2024-04-25T20:10:49.533145+0000","last_undegraded":"2024-04-25T20:10:49.533145+0000","last_fullsized":"2024-04-25T20:10:49.533145+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:41.384990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010123440000000001,"stat_sum":{"num_bytes":12615,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":46,"num_write":6,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"60'10","reported_seq":91,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T20:10:49.542798+0000","last_change":"2024-04-25T20:10:46.979954+0000","last_active":"2024-04-25T20:10:49.542798+0000","last_peered":"2024-04-25T20:10:49.542798+0000","last_clean":"2024-04-25T20:10:49.542798+0000","last_became_active":"2024-04-25T20:09:35.912678+0000","last_became_peered":"2024-04-25T20:09:35.912678+0000","last_unstale":"2024-04-25T20:10:49.542798+0000","last_undegraded":"2024-04-25T20:10:49.542798+0000","last_fullsized":"2024-04-25T20:10:49.542798+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:28.437601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029580399999999999,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":49,"num_read_kb":48,"num_write":8,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"60'26","reported_seq":176,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-25T20:10:49.354404+0000","last_change":"2024-04-25T20:10:46.983606+0000","last_active":"2024-04-25T20:10:49.354404+0000","last_peered":"2024-04-25T20:10:49.354404+0000","last_clean":"2024-04-25T20:10:49.354404+0000","last_became_active":"2024-04-25T20:09:16.140731+0000","last_became_peered":"2024-04-25T20:09:16.140731+0000","last_unstale":"2024-04-25T20:10:49.354404+0000","last_undegraded":"2024-04-25T20:10:49.354404+0000","last_fullsized":"2024-04-25T20:10:49.354404+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:18.458296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000216097,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":196,"num_read_kb":177,"num_write":25,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":123975,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":737,"num_read_kb":857,"num_write":136,"num_write_kb":255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":303104,"data_stored":247950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":851968,"data_stored":961777,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.371}]},{"osd":1,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.27400000000000002}]},{"osd":2,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.47399999999999998}]},{"osd":3,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.441}]},{"osd":4,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.46300000000000002}]},{"osd":5,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.40400000000000003}]},{"osd":6,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.48899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":757760,"data_stored":342017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.61099999999999999}]},{"osd":1,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.66000000000000003}]},{"osd":2,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.56399999999999995}]},{"osd":3,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.63}]},{"osd":4,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.64200000000000002}]},{"osd":5,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.68999999999999995}]},{"osd":7,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.65300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":811008,"data_stored":382394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.66100000000000003}]},{"osd":2,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.75900000000000001}]},{"osd":3,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.52300000000000002}]},{"osd":4,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.80700000000000005}]},{"osd":6,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.746},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.84099999999999997}]},{"osd":7,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.68200000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27724,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344116,"statfs":{"total":96636764160,"available":96608374784,"internally_reserved":0,"allocated":720896,"data_stored":301153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.27000000000000002}]},{"osd":1,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.375},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.28399999999999997}]},{"osd":3,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.41799999999999998}]},{"osd":5,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.312},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.32100000000000001}]},{"osd":6,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.245}]},{"osd":7,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.29999999999999999}]}]},{"osd":3,"up_from":0,"seq":55834574871,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31788,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":684032,"data_stored":282685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.40000000000000002}]},{"osd":1,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.23999999999999999}]},{"osd":3,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.42199999999999999}]},{"osd":4,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.55100000000000005}]},{"osd":5,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.52200000000000002}]},{"osd":6,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.46500000000000002}]},{"osd":7,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.503}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":811008,"data_stored":932369,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":3.0569999999999999,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":2.9660000000000002,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":0.23699999999999999}]},{"osd":2,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.0289999999999999,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.39400000000000002}]},{"osd":3,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.0859999999999999,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.9950000000000001,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":0.45800000000000002}]},{"osd":4,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.67800000000000005}]},{"osd":6,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.56999999999999995}]},{"osd":7,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.628},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.72899999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":729088,"data_stored":311109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Apr 25 20:10:15 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Apr 25 20:10:15 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.41299999999999998}]},{"osd":3,"last update":"Thu Apr 25 20:10:15 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.246},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.26700000000000002}]},{"osd":4,"last update":"Thu Apr 25 20:10:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.378},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.29999999999999999}]},{"osd":5,"last update":"Thu Apr 25 20:10:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.47299999999999998}]},{"osd":6,"last update":"Thu Apr 25 20:10:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.45600000000000002}]},{"osd":7,"last update":"Thu Apr 25 20:10:15 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.40200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":12631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":53479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":12631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":57865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":53495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":57849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:10:51.712 INFO:tasks.thrashosds.thrasher:Setting 1.2 to [1, 0, 7, 3] 2024-04-25T20:10:51.713 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '1.2', '1', '0', '7', '3'] 2024-04-25T20:10:51.713 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 1.2 1 0 7 3 2024-04-25T20:10:51.774 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:51.774+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:51.807 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:51.887 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:51.889 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-25T20:10:51.890 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:51.888+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:51.990 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:51.990+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:52.014 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is set 2024-04-25T20:10:52.024 INFO:tasks.workunit.client.0.smithi039.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2024-04-25T20:10:52.030 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd snap protect testrbdclonep47106@snap 2024-04-25T20:10:52.040 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:52.040 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:52.049 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-25T20:10:52.078 INFO:tasks.workunit.client.0.smithi039.stderr:+ local image=testrbdclone47106 2024-04-25T20:10:52.078 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd clone --image-feature layering --image-feature exclusive-lock --image-feature journaling --journal-pool rbd --journal-object-size 20M --journal-splay-width 6 testrbdclonep47106@snap testrbdclone47106 2024-04-25T20:10:52.091 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:52.091+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:52.154 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-25T20:10:52.163 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd_assert_eq testrbdclone47106 'journal info' //journal/order 25 2024-04-25T20:10:52.163 INFO:tasks.workunit.client.0.smithi039.stderr:+ local image=testrbdclone47106 2024-04-25T20:10:52.163 INFO:tasks.workunit.client.0.smithi039.stderr:+ local 'cmd=journal info' 2024-04-25T20:10:52.163 INFO:tasks.workunit.client.0.smithi039.stderr:+ local param=//journal/order 2024-04-25T20:10:52.164 INFO:tasks.workunit.client.0.smithi039.stderr:+ local expected_val=25 2024-04-25T20:10:52.164 INFO:tasks.workunit.client.0.smithi039.stderr:++ rbd --format xml journal info --image testrbdclone47106 2024-04-25T20:10:52.164 INFO:tasks.workunit.client.0.smithi039.stderr:++ xmlstarlet sel -t -v //journal/order 2024-04-25T20:10:52.189 INFO:teuthology.orchestra.run.smithi039.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:52.189 INFO:teuthology.orchestra.run.smithi039.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:52.192 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:52.192+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:52.200 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-25T20:10:52.227 INFO:tasks.workunit.client.0.smithi039.stderr:+ local val=25 2024-04-25T20:10:52.228 INFO:tasks.workunit.client.0.smithi039.stderr:+ test 25 = 25 2024-04-25T20:10:52.228 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd_assert_eq testrbdclone47106 'journal info' //journal/splay_width 6 2024-04-25T20:10:52.228 INFO:tasks.workunit.client.0.smithi039.stderr:+ local image=testrbdclone47106 2024-04-25T20:10:52.228 INFO:tasks.workunit.client.0.smithi039.stderr:+ local 'cmd=journal info' 2024-04-25T20:10:52.228 INFO:tasks.workunit.client.0.smithi039.stderr:+ local param=//journal/splay_width 2024-04-25T20:10:52.228 INFO:tasks.workunit.client.0.smithi039.stderr:+ local expected_val=6 2024-04-25T20:10:52.228 INFO:tasks.workunit.client.0.smithi039.stderr:++ rbd --format xml journal info --image testrbdclone47106 2024-04-25T20:10:52.228 INFO:tasks.workunit.client.0.smithi039.stderr:++ xmlstarlet sel -t -v //journal/splay_width 2024-04-25T20:10:52.289 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:10:52.289 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:10:52.293 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:52.292+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:52.293 INFO:tasks.workunit.client.0.smithi039.stderr:+ local val=6 2024-04-25T20:10:52.294 INFO:tasks.workunit.client.0.smithi039.stderr:+ test 6 = 6 2024-04-25T20:10:52.294 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd_assert_eq testrbdclone47106 'journal info' //journal/object_pool rbd 2024-04-25T20:10:52.294 INFO:tasks.workunit.client.0.smithi039.stderr:+ local image=testrbdclone47106 2024-04-25T20:10:52.294 INFO:tasks.workunit.client.0.smithi039.stderr:+ local 'cmd=journal info' 2024-04-25T20:10:52.294 INFO:tasks.workunit.client.0.smithi039.stderr:+ local param=//journal/object_pool 2024-04-25T20:10:52.294 INFO:tasks.workunit.client.0.smithi039.stderr:+ local expected_val=rbd 2024-04-25T20:10:52.294 INFO:tasks.workunit.client.0.smithi039.stderr:++ rbd --format xml journal info --image testrbdclone47106 2024-04-25T20:10:52.294 INFO:tasks.workunit.client.0.smithi039.stderr:++ xmlstarlet sel -t -v //journal/object_pool 2024-04-25T20:10:52.307 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:10:52.307 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:10:52.324 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:10:52.324 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:10:52.343 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:10:52.343 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:10:52.344 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:52.344 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:52.344 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:52.344 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:52.354 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-25T20:10:52.359 INFO:tasks.workunit.client.0.smithi039.stderr:+ local val=rbd 2024-04-25T20:10:52.359 INFO:tasks.workunit.client.0.smithi039.stderr:+ test rbd = rbd 2024-04-25T20:10:52.359 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd remove testrbdclone47106 2024-04-25T20:10:52.364 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:10:52.365 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:10:52.394 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:52.394+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:52.396 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:10:52.397 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:10:52.417 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:10:52.417 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:10:52.435 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:10:52.435 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:10:52.467 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:52.468 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:52.468 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:52.468 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:52.468 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:52.478 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-25T20:10:52.494 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:52.494+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:52.595 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:52.595+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:52.631 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:52.631 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:52.631 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:52.631 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-04-25T20:10:52.631 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.631 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:30 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'16, mlcod=16'14)", 2024-04-25T20:10:52.631 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:10:52.631 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 95.462554179999998, 2024-04-25T20:10:52.631 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0054087670000000001, 2024-04-25T20:10:52.631 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:52.631 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:52.631 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:52.632 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.632 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:52.632 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:10:52.632 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.632 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.632 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.632 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:52.632 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167144+0000", 2024-04-25T20:10:52.632 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.632 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.632 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.632 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:52.632 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167142+0000", 2024-04-25T20:10:52.632 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999976 2024-04-25T20:10:52.632 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.632 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.633 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:52.633 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167162+0000", 2024-04-25T20:10:52.633 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0109e-05 2024-04-25T20:10:52.633 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.633 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.633 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:52.633 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167188+0000", 2024-04-25T20:10:52.633 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6146e-05 2024-04-25T20:10:52.633 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.633 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.633 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:52.633 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167199+0000", 2024-04-25T20:10:52.633 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0767e-05 2024-04-25T20:10:52.633 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.633 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.634 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:52.634 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167256+0000", 2024-04-25T20:10:52.634 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7392000000000002e-05 2024-04-25T20:10:52.634 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.634 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.634 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:52.634 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167283+0000", 2024-04-25T20:10:52.634 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6577e-05 2024-04-25T20:10:52.634 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.634 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.634 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:52.634 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.172510+0000", 2024-04-25T20:10:52.634 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0052276759999999997 2024-04-25T20:10:52.634 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.635 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.635 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:52.635 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.172553+0000", 2024-04-25T20:10:52.635 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2444999999999998e-05 2024-04-25T20:10:52.635 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.635 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:52.635 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.635 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.635 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.635 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:31 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-04-25T20:10:52.635 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:52.635 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 95.461938102000005, 2024-04-25T20:10:52.635 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0055055479999999999, 2024-04-25T20:10:52.635 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:52.635 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:52.636 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:52.636 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.636 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:52.636 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:52.636 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.636 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.636 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.636 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:52.636 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:52.636 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.636 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.636 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.636 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:52.636 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167760+0000", 2024-04-25T20:10:52.636 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:52.637 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.637 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.637 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:52.637 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167766+0000", 2024-04-25T20:10:52.637 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.122e-06 2024-04-25T20:10:52.637 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.637 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.637 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:52.637 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167776+0000", 2024-04-25T20:10:52.637 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0515e-05 2024-04-25T20:10:52.637 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.637 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.637 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:52.637 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167782+0000", 2024-04-25T20:10:52.638 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1149999999999999e-06 2024-04-25T20:10:52.638 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.638 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.638 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:52.638 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167837+0000", 2024-04-25T20:10:52.638 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4763999999999999e-05 2024-04-25T20:10:52.638 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.638 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.638 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:52.638 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.167855+0000", 2024-04-25T20:10:52.638 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7589e-05 2024-04-25T20:10:52.638 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.638 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.638 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:52.639 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173221+0000", 2024-04-25T20:10:52.639 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.005365903 2024-04-25T20:10:52.639 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.639 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.639 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:52.639 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173266+0000", 2024-04-25T20:10:52.639 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5059999999999999e-05 2024-04-25T20:10:52.639 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.639 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:52.639 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.639 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.639 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.639 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:32 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-04-25T20:10:52.639 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:52.639 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 95.461562916999995, 2024-04-25T20:10:52.640 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0051558239999999998, 2024-04-25T20:10:52.640 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:52.640 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:52.640 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:52.640 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.640 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:52.640 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:52.640 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.640 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.640 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.640 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:52.640 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:52.640 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.640 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.640 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.641 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:52.641 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168135+0000", 2024-04-25T20:10:52.641 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:52.641 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.641 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.641 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:52.641 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168173+0000", 2024-04-25T20:10:52.641 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.7663999999999997e-05 2024-04-25T20:10:52.641 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.641 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.641 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:52.641 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168182+0000", 2024-04-25T20:10:52.641 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0650000000000005e-06 2024-04-25T20:10:52.641 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.641 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.642 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:52.642 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168188+0000", 2024-04-25T20:10:52.642 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0589999999999999e-06 2024-04-25T20:10:52.642 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.642 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.642 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:52.642 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168239+0000", 2024-04-25T20:10:52.642 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1467e-05 2024-04-25T20:10:52.642 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.642 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.642 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:52.642 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168260+0000", 2024-04-25T20:10:52.642 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0531e-05 2024-04-25T20:10:52.642 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.643 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.643 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:52.643 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173268+0000", 2024-04-25T20:10:52.643 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0050087980000000001 2024-04-25T20:10:52.643 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.643 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.643 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:52.643 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173291+0000", 2024-04-25T20:10:52.643 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.268e-05 2024-04-25T20:10:52.643 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.643 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:52.643 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.643 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.643 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.643 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:33 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'19, mlcod=16'14)", 2024-04-25T20:10:52.644 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:10:52.644 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 95.461052498000001, 2024-04-25T20:10:52.644 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046697049999999997, 2024-04-25T20:10:52.644 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:52.644 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:52.644 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:52.644 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.644 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:52.644 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:10:52.644 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.644 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.644 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.644 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:52.644 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168646+0000", 2024-04-25T20:10:52.644 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.645 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.645 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.645 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:52.645 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168645+0000", 2024-04-25T20:10:52.645 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:52.645 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.645 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.645 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:52.645 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168651+0000", 2024-04-25T20:10:52.645 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.1079999999999998e-06 2024-04-25T20:10:52.645 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.645 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.645 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:52.645 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168660+0000", 2024-04-25T20:10:52.646 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0010000000000007e-06 2024-04-25T20:10:52.646 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.646 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.646 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:52.646 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168666+0000", 2024-04-25T20:10:52.646 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7780000000000004e-06 2024-04-25T20:10:52.646 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.646 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.646 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:52.646 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168701+0000", 2024-04-25T20:10:52.646 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4490999999999999e-05 2024-04-25T20:10:52.646 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.646 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.646 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:52.647 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.168729+0000", 2024-04-25T20:10:52.647 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8476999999999999e-05 2024-04-25T20:10:52.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.647 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.647 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:52.647 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173294+0000", 2024-04-25T20:10:52.647 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0045650730000000002 2024-04-25T20:10:52.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.647 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.647 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:52.647 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173315+0000", 2024-04-25T20:10:52.647 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1245000000000001e-05 2024-04-25T20:10:52.647 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.647 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:52.647 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.648 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.648 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:34 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'20, mlcod=16'14)", 2024-04-25T20:10:52.648 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:10:52.648 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 95.460553568999998, 2024-04-25T20:10:52.648 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0041943309999999999, 2024-04-25T20:10:52.648 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:52.648 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:52.648 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:52.648 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.648 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:52.648 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:10:52.648 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.648 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.648 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.649 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:52.649 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169145+0000", 2024-04-25T20:10:52.649 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.649 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.649 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.649 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:52.649 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169144+0000", 2024-04-25T20:10:52.649 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:52.649 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.649 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.649 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:52.649 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169150+0000", 2024-04-25T20:10:52.649 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9150000000000001e-06 2024-04-25T20:10:52.649 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.649 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.650 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:52.650 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169159+0000", 2024-04-25T20:10:52.650 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8899999999999996e-06 2024-04-25T20:10:52.650 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.650 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.650 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:52.650 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169169+0000", 2024-04-25T20:10:52.650 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0441e-05 2024-04-25T20:10:52.650 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.650 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.650 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:52.650 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169206+0000", 2024-04-25T20:10:52.650 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6816000000000001e-05 2024-04-25T20:10:52.650 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.651 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.651 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:52.651 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169234+0000", 2024-04-25T20:10:52.651 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.8008999999999999e-05 2024-04-25T20:10:52.651 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.651 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.651 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:52.651 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173317+0000", 2024-04-25T20:10:52.651 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0040830160000000001 2024-04-25T20:10:52.651 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.651 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.651 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:52.651 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173339+0000", 2024-04-25T20:10:52.651 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1897000000000001e-05 2024-04-25T20:10:52.651 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.651 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:52.652 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.652 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.652 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.652 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:35 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'14)", 2024-04-25T20:10:52.652 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:10:52.652 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 95.460061249999995, 2024-04-25T20:10:52.652 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037236529999999999, 2024-04-25T20:10:52.652 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:52.652 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:52.652 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:52.652 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.652 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:52.652 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:10:52.652 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.652 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.653 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.653 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:52.653 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169637+0000", 2024-04-25T20:10:52.653 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.653 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.653 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.653 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:52.653 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169636+0000", 2024-04-25T20:10:52.653 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:52.653 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.653 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.653 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:52.653 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169678+0000", 2024-04-25T20:10:52.653 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.1752000000000003e-05 2024-04-25T20:10:52.653 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.654 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.654 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:52.654 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169688+0000", 2024-04-25T20:10:52.654 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0098000000000001e-05 2024-04-25T20:10:52.654 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.654 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.654 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:52.654 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169694+0000", 2024-04-25T20:10:52.654 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2349999999999996e-06 2024-04-25T20:10:52.654 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.654 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.654 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:52.654 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169733+0000", 2024-04-25T20:10:52.654 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8481e-05 2024-04-25T20:10:52.654 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.655 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.655 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:52.655 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.169750+0000", 2024-04-25T20:10:52.655 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7509000000000001e-05 2024-04-25T20:10:52.655 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.655 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.655 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:52.655 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173340+0000", 2024-04-25T20:10:52.655 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0035898449999999999 2024-04-25T20:10:52.655 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.655 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.655 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:52.655 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173361+0000", 2024-04-25T20:10:52.655 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.039e-05 2024-04-25T20:10:52.655 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.656 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:52.656 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.656 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.656 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.656 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:36 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'22, mlcod=16'14)", 2024-04-25T20:10:52.656 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:52.656 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 95.459561190000002, 2024-04-25T20:10:52.656 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0032472120000000002, 2024-04-25T20:10:52.656 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:52.656 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:52.656 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:52.656 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.656 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:52.656 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:52.657 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.657 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.657 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.657 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:52.657 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:52.657 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.657 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.657 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.657 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:52.657 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170137+0000", 2024-04-25T20:10:52.657 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:52.657 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.657 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.657 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:52.658 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170152+0000", 2024-04-25T20:10:52.658 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5798e-05 2024-04-25T20:10:52.658 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.658 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.658 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:52.658 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170161+0000", 2024-04-25T20:10:52.658 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.064e-06 2024-04-25T20:10:52.658 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.658 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.658 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:52.658 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170167+0000", 2024-04-25T20:10:52.658 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8390000000000003e-06 2024-04-25T20:10:52.658 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.658 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.659 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:52.659 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170202+0000", 2024-04-25T20:10:52.659 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.5120999999999998e-05 2024-04-25T20:10:52.659 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.659 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.659 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:52.659 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.170219+0000", 2024-04-25T20:10:52.659 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6929999999999999e-05 2024-04-25T20:10:52.659 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.659 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.659 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:52.659 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173362+0000", 2024-04-25T20:10:52.659 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0031427619999999999 2024-04-25T20:10:52.659 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.659 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.660 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:52.660 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.173384+0000", 2024-04-25T20:10:52.660 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2237000000000001e-05 2024-04-25T20:10:52.660 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.660 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:52.660 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.660 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.660 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.660 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:67 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'41, mlcod=16'39)", 2024-04-25T20:10:52.660 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:10:52.660 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 95.417761786, 2024-04-25T20:10:52.660 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064192549999999996, 2024-04-25T20:10:52.660 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:52.660 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:52.661 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:52.661 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.661 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:52.661 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:10:52.661 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.661 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.661 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.661 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:52.661 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211936+0000", 2024-04-25T20:10:52.661 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.661 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.661 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.661 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:52.661 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211935+0000", 2024-04-25T20:10:52.661 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999981 2024-04-25T20:10:52.662 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.662 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.662 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:52.662 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211948+0000", 2024-04-25T20:10:52.662 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3531e-05 2024-04-25T20:10:52.662 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.662 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.662 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:52.662 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211967+0000", 2024-04-25T20:10:52.662 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8326999999999999e-05 2024-04-25T20:10:52.662 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.662 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.662 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:52.662 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.211975+0000", 2024-04-25T20:10:52.663 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8589999999999994e-06 2024-04-25T20:10:52.663 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.663 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.663 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:52.663 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212020+0000", 2024-04-25T20:10:52.663 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4626999999999998e-05 2024-04-25T20:10:52.663 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.663 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.663 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:52.663 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212045+0000", 2024-04-25T20:10:52.663 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4508999999999999e-05 2024-04-25T20:10:52.663 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.663 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.663 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:52.663 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218316+0000", 2024-04-25T20:10:52.664 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0062711640000000001 2024-04-25T20:10:52.664 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.664 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.664 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:52.664 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218356+0000", 2024-04-25T20:10:52.664 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.9993000000000001e-05 2024-04-25T20:10:52.664 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.664 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:52.664 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.664 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.664 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.664 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:68 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-04-25T20:10:52.664 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:52.664 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 95.417312772000002, 2024-04-25T20:10:52.664 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011231365, 2024-04-25T20:10:52.665 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:52.665 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:52.665 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:52.665 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.665 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:52.665 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:52.665 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.665 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.665 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.665 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:52.665 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:52.665 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.665 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.665 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.665 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:52.666 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212385+0000", 2024-04-25T20:10:52.666 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:52.666 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.666 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.666 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:52.666 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212392+0000", 2024-04-25T20:10:52.666 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.79e-06 2024-04-25T20:10:52.666 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.666 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.666 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:52.666 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212406+0000", 2024-04-25T20:10:52.666 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3577000000000001e-05 2024-04-25T20:10:52.666 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.666 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.666 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:52.667 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212414+0000", 2024-04-25T20:10:52.667 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6510000000000001e-06 2024-04-25T20:10:52.667 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.667 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.667 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:52.667 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217114+0000", 2024-04-25T20:10:52.667 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047005379999999998 2024-04-25T20:10:52.667 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.667 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.667 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:52.667 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217141+0000", 2024-04-25T20:10:52.667 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6984000000000001e-05 2024-04-25T20:10:52.667 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.667 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.667 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:52.668 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223563+0000", 2024-04-25T20:10:52.668 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0064216999999999998 2024-04-25T20:10:52.668 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.668 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.668 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:52.668 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223617+0000", 2024-04-25T20:10:52.668 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.4011000000000002e-05 2024-04-25T20:10:52.668 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.668 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:52.668 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.668 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.668 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.668 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:69 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-04-25T20:10:52.668 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:10:52.668 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 95.416780582000001, 2024-04-25T20:10:52.669 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.010730094000000001, 2024-04-25T20:10:52.669 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:52.669 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:52.669 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:52.669 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.669 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:52.669 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:10:52.669 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.669 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.669 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.669 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:52.669 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212918+0000", 2024-04-25T20:10:52.669 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.669 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.670 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.670 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:52.670 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212917+0000", 2024-04-25T20:10:52.670 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:52.670 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.670 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.670 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:52.670 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212957+0000", 2024-04-25T20:10:52.670 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.0692000000000003e-05 2024-04-25T20:10:52.670 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.670 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.670 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:52.670 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212973+0000", 2024-04-25T20:10:52.670 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.575e-05 2024-04-25T20:10:52.671 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.671 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.671 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:52.671 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.212980+0000", 2024-04-25T20:10:52.671 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1679999999999998e-06 2024-04-25T20:10:52.671 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.671 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.671 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:52.671 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217449+0000", 2024-04-25T20:10:52.671 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.004468367 2024-04-25T20:10:52.671 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.671 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.671 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:52.671 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.217470+0000", 2024-04-25T20:10:52.671 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.0749000000000001e-05 2024-04-25T20:10:52.671 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.672 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.672 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:52.672 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223621+0000", 2024-04-25T20:10:52.672 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0061512880000000004 2024-04-25T20:10:52.672 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.672 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.672 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:52.672 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.223648+0000", 2024-04-25T20:10:52.672 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.6961000000000002e-05 2024-04-25T20:10:52.672 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.672 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:52.672 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.672 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.672 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.672 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:70 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'44, mlcod=16'39)", 2024-04-25T20:10:52.672 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:52.673 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 95.416021853999993, 2024-04-25T20:10:52.673 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.013553358, 2024-04-25T20:10:52.673 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:52.673 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:52.673 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:52.673 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.673 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:52.673 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:52.673 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.673 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.673 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.673 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:52.673 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:52.673 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.673 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.673 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.673 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:52.674 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213676+0000", 2024-04-25T20:10:52.674 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:52.674 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.674 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.674 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:52.674 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213682+0000", 2024-04-25T20:10:52.674 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.3439999999999997e-06 2024-04-25T20:10:52.674 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.674 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.674 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:52.674 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213693+0000", 2024-04-25T20:10:52.674 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1049999999999999e-05 2024-04-25T20:10:52.674 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.674 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.675 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:52.675 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.213699+0000", 2024-04-25T20:10:52.675 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9399999999999999e-06 2024-04-25T20:10:52.675 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.675 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.675 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:52.675 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218086+0000", 2024-04-25T20:10:52.675 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0043871240000000001 2024-04-25T20:10:52.675 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.675 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.675 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:52.675 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.218106+0000", 2024-04-25T20:10:52.675 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.997e-05 2024-04-25T20:10:52.675 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.675 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.676 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:52.676 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.227186+0000", 2024-04-25T20:10:52.676 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.009079574 2024-04-25T20:10:52.676 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.676 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.676 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:52.676 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.227230+0000", 2024-04-25T20:10:52.676 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4147999999999998e-05 2024-04-25T20:10:52.676 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.676 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:52.676 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.676 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.676 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.676 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:106 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-04-25T20:10:52.676 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:10:52.677 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 95.044882754, 2024-04-25T20:10:52.677 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002936756, 2024-04-25T20:10:52.677 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:52.677 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:52.677 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:52.677 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.677 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:52.677 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:10:52.677 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.677 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.677 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.677 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:52.677 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584816+0000", 2024-04-25T20:10:52.677 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.677 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.678 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.678 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:52.678 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584815+0000", 2024-04-25T20:10:52.678 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:52.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.678 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.678 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:52.678 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584860+0000", 2024-04-25T20:10:52.678 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.5161999999999999e-05 2024-04-25T20:10:52.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.678 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.678 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:52.678 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584874+0000", 2024-04-25T20:10:52.678 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4464000000000001e-05 2024-04-25T20:10:52.679 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.679 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.679 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:52.679 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584881+0000", 2024-04-25T20:10:52.679 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.409e-06 2024-04-25T20:10:52.679 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.679 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.679 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:52.679 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584915+0000", 2024-04-25T20:10:52.679 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4007000000000002e-05 2024-04-25T20:10:52.679 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.679 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.679 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:52.681 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.584933+0000", 2024-04-25T20:10:52.681 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7737999999999999e-05 2024-04-25T20:10:52.681 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.681 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.681 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:52.681 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587717+0000", 2024-04-25T20:10:52.681 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.002784711 2024-04-25T20:10:52.681 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.681 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.681 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:52.681 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587752+0000", 2024-04-25T20:10:52.682 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.4915e-05 2024-04-25T20:10:52.682 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.682 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:52.682 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.682 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.682 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:112 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'74, mlcod=16'66)", 2024-04-25T20:10:52.682 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:10:52.682 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 95.042624678999999, 2024-04-25T20:10:52.682 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00318724, 2024-04-25T20:10:52.682 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:52.682 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:52.682 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:52.682 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.683 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:52.683 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:10:52.683 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.683 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.683 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:52.683 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587074+0000", 2024-04-25T20:10:52.683 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.683 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.683 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:52.683 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587073+0000", 2024-04-25T20:10:52.683 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:52.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.683 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.684 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:52.684 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587136+0000", 2024-04-25T20:10:52.684 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2928999999999993e-05 2024-04-25T20:10:52.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.684 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.684 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:52.684 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587147+0000", 2024-04-25T20:10:52.684 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1182999999999999e-05 2024-04-25T20:10:52.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.684 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.684 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:52.684 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587153+0000", 2024-04-25T20:10:52.684 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.9059999999999999e-06 2024-04-25T20:10:52.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.684 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.684 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:52.685 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587306+0000", 2024-04-25T20:10:52.685 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000152848 2024-04-25T20:10:52.685 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.685 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.685 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:52.685 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.587330+0000", 2024-04-25T20:10:52.685 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4304e-05 2024-04-25T20:10:52.685 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.685 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.685 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:52.685 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.590230+0000", 2024-04-25T20:10:52.685 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029004629999999998 2024-04-25T20:10:52.685 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.685 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.685 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:52.686 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.590261+0000", 2024-04-25T20:10:52.686 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.0323000000000001e-05 2024-04-25T20:10:52.686 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.686 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:52.686 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.686 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.686 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.686 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:118 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'80, mlcod=16'78)", 2024-04-25T20:10:52.686 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:10:52.686 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 95.037838100000002, 2024-04-25T20:10:52.686 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029618959999999999, 2024-04-25T20:10:52.686 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:52.686 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:52.686 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:52.687 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.687 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:52.687 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:10:52.687 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.687 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.687 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:52.687 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591860+0000", 2024-04-25T20:10:52.687 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.687 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.687 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:52.687 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591859+0000", 2024-04-25T20:10:52.687 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.999999 2024-04-25T20:10:52.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.688 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.688 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:52.688 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591918+0000", 2024-04-25T20:10:52.688 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.8842999999999999e-05 2024-04-25T20:10:52.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.688 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.688 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:52.688 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591933+0000", 2024-04-25T20:10:52.688 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.4600000000000001e-05 2024-04-25T20:10:52.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.688 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.688 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:52.688 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591939+0000", 2024-04-25T20:10:52.688 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4559999999999997e-06 2024-04-25T20:10:52.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.689 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.689 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:52.689 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.591990+0000", 2024-04-25T20:10:52.689 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.0318999999999997e-05 2024-04-25T20:10:52.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.689 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.689 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:52.689 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.592008+0000", 2024-04-25T20:10:52.689 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8819000000000001e-05 2024-04-25T20:10:52.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.689 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.689 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:52.689 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.594784+0000", 2024-04-25T20:10:52.689 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0027750850000000001 2024-04-25T20:10:52.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.689 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.690 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:52.690 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:17.594822+0000", 2024-04-25T20:10:52.690 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.8522000000000001e-05 2024-04-25T20:10:52.690 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.690 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:52.690 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.690 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.690 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:139 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'97, mlcod=17'87)", 2024-04-25T20:10:52.690 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:10:52.690 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 94.503801495000005, 2024-04-25T20:10:52.690 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048546290000000001, 2024-04-25T20:10:52.690 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:52.690 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:52.691 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:52.691 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.691 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:52.691 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:10:52.691 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.691 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.691 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.691 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:52.691 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125897+0000", 2024-04-25T20:10:52.691 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.691 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.691 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.691 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:52.691 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125896+0000", 2024-04-25T20:10:52.691 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:52.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.692 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.692 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:52.692 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125908+0000", 2024-04-25T20:10:52.692 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.137e-05 2024-04-25T20:10:52.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.692 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.692 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:52.692 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125915+0000", 2024-04-25T20:10:52.692 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.3939999999999998e-06 2024-04-25T20:10:52.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.692 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.692 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:52.692 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125922+0000", 2024-04-25T20:10:52.692 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1930000000000004e-06 2024-04-25T20:10:52.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.693 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.693 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:52.693 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125959+0000", 2024-04-25T20:10:52.693 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6701e-05 2024-04-25T20:10:52.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.693 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.693 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:52.693 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.125972+0000", 2024-04-25T20:10:52.693 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2578000000000001e-05 2024-04-25T20:10:52.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.693 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.693 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:52.693 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130719+0000", 2024-04-25T20:10:52.693 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0047475800000000004 2024-04-25T20:10:52.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.693 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.694 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:52.694 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130751+0000", 2024-04-25T20:10:52.694 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2283000000000002e-05 2024-04-25T20:10:52.694 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.694 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:52.694 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.694 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.694 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.694 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:140 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'98, mlcod=17'87)", 2024-04-25T20:10:52.694 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:52.694 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 94.503376144000001, 2024-04-25T20:10:52.694 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0048310990000000002, 2024-04-25T20:10:52.694 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:52.694 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:52.694 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:52.695 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.695 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:52.695 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:52.695 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.695 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.695 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:52.695 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:52.695 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.695 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.695 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:52.695 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126322+0000", 2024-04-25T20:10:52.695 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:52.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.695 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.696 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:52.696 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126427+0000", 2024-04-25T20:10:52.696 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000105572 2024-04-25T20:10:52.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.696 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.696 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:52.696 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126435+0000", 2024-04-25T20:10:52.696 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.1920000000000005e-06 2024-04-25T20:10:52.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.696 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.696 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:52.696 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126439+0000", 2024-04-25T20:10:52.696 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.6950000000000002e-06 2024-04-25T20:10:52.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.697 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.697 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:52.697 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126471+0000", 2024-04-25T20:10:52.697 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.1699000000000003e-05 2024-04-25T20:10:52.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.697 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.697 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:52.697 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126484+0000", 2024-04-25T20:10:52.697 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.343e-05 2024-04-25T20:10:52.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.697 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.697 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:52.697 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131131+0000", 2024-04-25T20:10:52.697 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046471480000000003 2024-04-25T20:10:52.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.697 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.698 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:52.698 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131153+0000", 2024-04-25T20:10:52.698 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1753e-05 2024-04-25T20:10:52.698 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.698 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:52.698 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.698 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.698 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4175.0:141 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'99, mlcod=17'87)", 2024-04-25T20:10:52.698 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:52.698 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 94.502721199999996, 2024-04-25T20:10:52.698 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.004496847, 2024-04-25T20:10:52.698 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:52.698 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:52.699 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:52.699 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.699 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:52.699 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:52.699 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.699 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.699 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.699 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:52.699 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:52.699 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.699 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.699 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.699 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:52.699 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126977+0000", 2024-04-25T20:10:52.699 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:52.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.700 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.700 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:52.700 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126980+0000", 2024-04-25T20:10:52.700 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.619e-06 2024-04-25T20:10:52.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.700 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.700 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:52.700 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126987+0000", 2024-04-25T20:10:52.700 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.7220000000000002e-06 2024-04-25T20:10:52.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.700 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.700 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:52.700 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.126992+0000", 2024-04-25T20:10:52.700 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.8679999999999998e-06 2024-04-25T20:10:52.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.701 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.701 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:52.701 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130772+0000", 2024-04-25T20:10:52.701 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037803120000000001 2024-04-25T20:10:52.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.701 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.701 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:52.701 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.130786+0000", 2024-04-25T20:10:52.701 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3592e-05 2024-04-25T20:10:52.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.701 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.701 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:52.701 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131451+0000", 2024-04-25T20:10:52.701 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.000665206 2024-04-25T20:10:52.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.701 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.702 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:52.702 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:09:18.131474+0000", 2024-04-25T20:10:52.702 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2957999999999999e-05 2024-04-25T20:10:52.702 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.702 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:52.702 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.702 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.702 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4982.0:9 1.7 e59/55 1:e33cb61b:::journal_data.1.145ee3bcd187.2:head v 59'1, mlcod=0'0)", 2024-04-25T20:10:52.702 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:48.400129+0000", 2024-04-25T20:10:52.702 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.2295690769999998, 2024-04-25T20:10:52.702 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0058907949999999999, 2024-04-25T20:10:52.702 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:52.702 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:52.703 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:52.703 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.703 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:52.703 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400129+0000", 2024-04-25T20:10:52.703 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.703 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.703 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:52.703 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400129+0000", 2024-04-25T20:10:52.703 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.703 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.703 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:52.703 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400128+0000", 2024-04-25T20:10:52.703 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999986 2024-04-25T20:10:52.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.704 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.704 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:52.704 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400143+0000", 2024-04-25T20:10:52.704 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5358000000000001e-05 2024-04-25T20:10:52.704 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.704 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.704 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:52.704 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400161+0000", 2024-04-25T20:10:52.704 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7280999999999999e-05 2024-04-25T20:10:52.704 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.704 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.704 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:52.704 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400170+0000", 2024-04-25T20:10:52.704 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.2299999999999997e-06 2024-04-25T20:10:52.704 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.705 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.705 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:52.705 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400231+0000", 2024-04-25T20:10:52.705 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.0902000000000001e-05 2024-04-25T20:10:52.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.705 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.705 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:52.705 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400249+0000", 2024-04-25T20:10:52.705 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7977000000000001e-05 2024-04-25T20:10:52.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.705 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.705 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:52.705 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.405996+0000", 2024-04-25T20:10:52.705 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.005747355 2024-04-25T20:10:52.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.705 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.705 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:52.706 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.406020+0000", 2024-04-25T20:10:52.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3952e-05 2024-04-25T20:10:52.706 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.706 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:52.706 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.706 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.706 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4982.0:13 1.7 e59/55 1:e33cb61b:::journal_data.1.145ee3bcd187.2:head v 59'2, mlcod=0'0)", 2024-04-25T20:10:52.706 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:48.400761+0000", 2024-04-25T20:10:52.706 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.2289367609999999, 2024-04-25T20:10:52.706 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0057307879999999997, 2024-04-25T20:10:52.706 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:52.706 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:52.706 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:52.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:52.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400761+0000", 2024-04-25T20:10:52.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:52.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400761+0000", 2024-04-25T20:10:52.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.707 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.707 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:52.707 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400761+0000", 2024-04-25T20:10:52.707 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:52.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.708 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:52.708 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400770+0000", 2024-04-25T20:10:52.708 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7129999999999994e-06 2024-04-25T20:10:52.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.708 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:52.708 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400781+0000", 2024-04-25T20:10:52.708 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1255e-05 2024-04-25T20:10:52.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.708 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:52.708 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400789+0000", 2024-04-25T20:10:52.708 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8550000000000002e-06 2024-04-25T20:10:52.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.709 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:52.709 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400822+0000", 2024-04-25T20:10:52.709 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3340000000000003e-05 2024-04-25T20:10:52.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.709 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:52.709 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.400834+0000", 2024-04-25T20:10:52.709 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1377e-05 2024-04-25T20:10:52.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.709 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:52.709 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.406459+0000", 2024-04-25T20:10:52.709 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0056257829999999997 2024-04-25T20:10:52.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.710 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:52.710 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.406492+0000", 2024-04-25T20:10:52.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2926999999999998e-05 2024-04-25T20:10:52.710 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.710 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:52.710 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.710 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.4982.0:17 1.7 e59/55 1:e33cb61b:::journal_data.1.145ee3bcd187.2:head v 59'3, mlcod=0'0)", 2024-04-25T20:10:52.710 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-04-25T20:10:48.401348+0000", 2024-04-25T20:10:52.710 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 4.2283499039999999, 2024-04-25T20:10:52.710 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0053994259999999997, 2024-04-25T20:10:52.710 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-04-25T20:10:52.711 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:52.711 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-04-25T20:10:52.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.711 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-04-25T20:10:52.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.401348+0000", 2024-04-25T20:10:52.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.711 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-04-25T20:10:52.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.401348+0000", 2024-04-25T20:10:52.711 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-04-25T20:10:52.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.711 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.711 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-04-25T20:10:52.711 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.401348+0000", 2024-04-25T20:10:52.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:52.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.712 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.712 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-04-25T20:10:52.712 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.401357+0000", 2024-04-25T20:10:52.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.1919999999999994e-06 2024-04-25T20:10:52.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.712 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.712 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-04-25T20:10:52.712 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.401365+0000", 2024-04-25T20:10:52.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6260000000000003e-06 2024-04-25T20:10:52.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.712 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.712 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-04-25T20:10:52.712 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.401370+0000", 2024-04-25T20:10:52.712 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.5520000000000004e-06 2024-04-25T20:10:52.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.713 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-04-25T20:10:52.713 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.406055+0000", 2024-04-25T20:10:52.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0046851050000000002 2024-04-25T20:10:52.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.713 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-04-25T20:10:52.713 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.406077+0000", 2024-04-25T20:10:52.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1078999999999999e-05 2024-04-25T20:10:52.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.713 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.713 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "commit_sent", 2024-04-25T20:10:52.713 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.406721+0000", 2024-04-25T20:10:52.713 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00064424999999999997 2024-04-25T20:10:52.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-04-25T20:10:52.714 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-04-25T20:10:52.714 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "done", 2024-04-25T20:10:52.714 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-04-25T20:10:48.406748+0000", 2024-04-25T20:10:52.714 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7012000000000002e-05 2024-04-25T20:10:52.714 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.714 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:52.714 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.714 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-04-25T20:10:52.714 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-04-25T20:10:52.714 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:52.716 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-25T20:10:52.721 INFO:tasks.workunit.client.0.smithi039.stderr: Removing image: 1% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 9% complete...2024-04-25T20:10:52.653+0000 7fc674185640 0 -- 172.21.15.39:0/517788662 >> [v2:172.21.15.39:6816/31832,v1:172.21.15.39:6818/31832] conn(0x564f9ece1280 msgr2=0x564f9ece3710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-25T20:10:52.721 INFO:tasks.workunit.client.0.smithi039.stderr: Removing image: 10% complete... Removing image: 12% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 23% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 37% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 42% complete... Removing image: 43% complete...2024-04-25T20:10:52.658+0000 7fc6753e7640 0 -- 172.21.15.39:0/517788662 >> [v2:172.21.15.135:6808/31725,v1:172.21.15.135:6809/31725] conn(0x7fc650016920 msgr2=0x7fc650019da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-25T20:10:52.721 INFO:tasks.workunit.client.0.smithi039.stderr: Removing image: 45% complete... Removing image: 46% complete... Removing image: 48% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 62% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 73% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 87% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 98% complete... Removing image: 100% complete...done. 2024-04-25T20:10:52.722 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd snap unprotect testrbdclonep47106@snap 2024-04-25T20:10:52.722 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:52.715+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:52.746 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd snap purge testrbdclonep47106 2024-04-25T20:10:52.817 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:52.816+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:52.852 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:52.853 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:52.853 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:52.853 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:52.861 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-25T20:10:52.945 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:52.945+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:53.007 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:53.008 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:53.008 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:53.008 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:53.008 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:53.016 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-25T20:10:53.035 INFO:teuthology.orchestra.run.smithi039.stderr:set 1.2 pg_upmap_items mapping to [1->0,7->3] 2024-04-25T20:10:53.036 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:53.035+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:53.039 INFO:tasks.workunit.client.0.smithi039.stderr: Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2024-04-25T20:10:53.044 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd remove testrbdclonep47106 2024-04-25T20:10:53.118 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:53.118+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:53.125 INFO:tasks.workunit.client.0.smithi039.stderr: Removing image: 1% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 12% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 23% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 37% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 48% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 56% complete... Removing image: 57% complete...2024-04-25T20:10:53.124+0000 7f6207195640 0 -- 172.21.15.39:0/3099364282 >> [v2:172.21.15.39:6816/31832,v1:172.21.15.39:6818/31832] conn(0x56202a143050 msgr2=0x56202a1454e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-25T20:10:53.130 INFO:tasks.workunit.client.0.smithi039.stderr: Removing image: 59% complete... Removing image: 60% complete... Removing image: 62% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 73% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 87% complete... Removing image: 89% complete...2024-04-25T20:10:53.129+0000 7f6206994640 0 -- 172.21.15.39:0/3099364282 >> [v2:172.21.15.135:6808/31725,v1:172.21.15.135:6809/31725] conn(0x7f61e40136b0 msgr2=0x7f61e4016600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-25T20:10:53.140 INFO:tasks.workunit.client.0.smithi039.stderr: Removing image: 90% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 98% complete... Removing image: 100% complete...done. 2024-04-25T20:10:53.143 INFO:tasks.workunit.client.0.smithi039.stderr:+ set +x 2024-04-25T20:10:53.144 INFO:tasks.workunit.client.0.smithi039.stderr:+ test_rbd_import 2024-04-25T20:10:53.144 INFO:tasks.workunit.client.0.smithi039.stderr:+ local src=testrbdimports47106 2024-04-25T20:10:53.144 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd create --size 256 testrbdimports47106 2024-04-25T20:10:53.156 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:53.156 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:53.156 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:53.156 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:10:53.156 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:53.166 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-25T20:10:53.196 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd export testrbdimports47106 /tmp/rbd_journal47106/testrbdimports47106.export 2024-04-25T20:10:53.219 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:53.219+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:53.276 INFO:tasks.workunit.client.0.smithi039.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2024-04-25T20:10:53.281 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd remove testrbdimports47106 2024-04-25T20:10:53.317 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:53.317 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:53.317 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-04-25T20:10:53.317 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:53.320 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:53.320+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:53.326 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-25T20:10:53.366 INFO:tasks.workunit.client.0.smithi039.stderr:2024-04-25T20:10:53.365+0000 7f26c9e33640 0 -- 172.21.15.39:0/3584532172 >> [v2:172.21.15.135:6816/31737,v1:172.21.15.135:6817/31737] conn(0x5615dac6b650 msgr2=0x5615dac6da40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-25T20:10:53.377 INFO:tasks.workunit.client.0.smithi039.stderr: Removing image: 1% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 12% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 23% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 37% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 48% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 62% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 73% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 87% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 98% complete...2024-04-25T20:10:53.376+0000 7f26c3fff640 0 -- 172.21.15.39:0/3584532172 >> [v2:172.21.15.135:6800/31724,v1:172.21.15.135:6801/31724] conn(0x7f26a8006cc0 msgr2=0x7f26a8009180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-25T20:10:53.379 INFO:tasks.workunit.client.0.smithi039.stderr: Removing image: 100% complete...done. 2024-04-25T20:10:53.382 INFO:tasks.workunit.client.0.smithi039.stderr:+ local image=testrbdimport47106 2024-04-25T20:10:53.382 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd import --image-feature exclusive-lock --image-feature journaling --journal-pool rbd --journal-object-size 20M --journal-splay-width 6 /tmp/rbd_journal47106/testrbdimports47106.export testrbdimport47106 2024-04-25T20:10:53.421 INFO:tasks.ceph.osd.4.smithi135.stderr:2024-04-25T20:10:53.421+0000 7f34fbcde640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31714) UID: 0 2024-04-25T20:10:53.477 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:53.477 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-04-25T20:10:53.478 INFO:teuthology.orchestra.run.smithi039.stdout: "complaint_time": 30, 2024-04-25T20:10:53.478 INFO:teuthology.orchestra.run.smithi039.stdout: "num_blocked_ops": 0 2024-04-25T20:10:53.478 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:53.490 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-25T20:10:53.521 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:53.521+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:53.572 INFO:tasks.workunit.client.0.smithi039.stderr: Importing image: 1% complete... Importing image: 3% complete... Importing image: 4% complete... Importing image: 6% complete... Importing image: 7% complete... Importing image: 9% complete... Importing image: 10% complete... Importing image: 12% complete... Importing image: 14% complete... Importing image: 15% complete... Importing image: 17% complete... Importing image: 18% complete... Importing image: 20% complete... Importing image: 21% complete... Importing image: 23% complete... Importing image: 25% complete... Importing image: 26% complete... Importing image: 28% complete... Importing image: 29% complete... Importing image: 31% complete... Importing image: 32% complete... Importing image: 34% complete... Importing image: 35% complete... Importing image: 37% complete... Importing image: 39% complete... Importing image: 40% complete... Importing image: 42% complete... Importing image: 43% complete... Importing image: 45% complete... Importing image: 46% complete... Importing image: 48% complete... Importing image: 50% complete... Importing image: 51% complete... Importing image: 53% complete... Importing image: 54% complete... Importing image: 56% complete... Importing image: 57% complete... Importing image: 59% complete... Importing image: 60% complete... Importing image: 62% complete... Importing image: 64% complete... Importing image: 65% complete... Importing image: 67% complete... Importing image: 68% complete... Importing image: 70% complete... Importing image: 71% complete... Importing image: 73% complete... Importing image: 75% complete... Importing image: 76% complete... Importing image: 78% complete... Importing image: 79% complete... Importing image: 81% complete... Importing image: 82% complete... Importing image: 84% complete... Importing image: 85% complete... Importing image: 87% complete... Importing image: 89% complete... Importing image: 90% complete... Importing image: 92% complete... Importing image: 93% complete... Importing image: 95% complete... Importing image: 96% complete... Importing image: 98% complete... Importing image: 100% complete...done. 2024-04-25T20:10:53.576 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd_assert_eq testrbdimport47106 'journal info' //journal/order 25 2024-04-25T20:10:53.576 INFO:tasks.workunit.client.0.smithi039.stderr:+ local image=testrbdimport47106 2024-04-25T20:10:53.576 INFO:tasks.workunit.client.0.smithi039.stderr:+ local 'cmd=journal info' 2024-04-25T20:10:53.576 INFO:tasks.workunit.client.0.smithi039.stderr:+ local param=//journal/order 2024-04-25T20:10:53.576 INFO:tasks.workunit.client.0.smithi039.stderr:+ local expected_val=25 2024-04-25T20:10:53.577 INFO:tasks.workunit.client.0.smithi039.stderr:++ rbd --format xml journal info --image testrbdimport47106 2024-04-25T20:10:53.577 INFO:tasks.workunit.client.0.smithi039.stderr:++ xmlstarlet sel -t -v //journal/order 2024-04-25T20:10:53.612 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:10:53.612 INFO:teuthology.orchestra.run.smithi039.stdout: "size": 20, 2024-04-25T20:10:53.612 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 600, 2024-04-25T20:10:53.612 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [] 2024-04-25T20:10:53.612 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:10:53.622 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-25T20:10:53.623 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:53.622+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:53.632 INFO:tasks.workunit.client.0.smithi039.stderr:+ local val=25 2024-04-25T20:10:53.632 INFO:tasks.workunit.client.0.smithi039.stderr:+ test 25 = 25 2024-04-25T20:10:53.632 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd_assert_eq testrbdimport47106 'journal info' //journal/splay_width 6 2024-04-25T20:10:53.632 INFO:tasks.workunit.client.0.smithi039.stderr:+ local image=testrbdimport47106 2024-04-25T20:10:53.632 INFO:tasks.workunit.client.0.smithi039.stderr:+ local 'cmd=journal info' 2024-04-25T20:10:53.632 INFO:tasks.workunit.client.0.smithi039.stderr:+ local param=//journal/splay_width 2024-04-25T20:10:53.633 INFO:tasks.workunit.client.0.smithi039.stderr:+ local expected_val=6 2024-04-25T20:10:53.633 INFO:tasks.workunit.client.0.smithi039.stderr:++ rbd --format xml journal info --image testrbdimport47106 2024-04-25T20:10:53.633 INFO:tasks.workunit.client.0.smithi039.stderr:++ xmlstarlet sel -t -v //journal/splay_width 2024-04-25T20:10:53.684 INFO:tasks.workunit.client.0.smithi039.stderr:+ local val=6 2024-04-25T20:10:53.684 INFO:tasks.workunit.client.0.smithi039.stderr:+ test 6 = 6 2024-04-25T20:10:53.684 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd_assert_eq testrbdimport47106 'journal info' //journal/object_pool rbd 2024-04-25T20:10:53.684 INFO:tasks.workunit.client.0.smithi039.stderr:+ local image=testrbdimport47106 2024-04-25T20:10:53.684 INFO:tasks.workunit.client.0.smithi039.stderr:+ local 'cmd=journal info' 2024-04-25T20:10:53.684 INFO:tasks.workunit.client.0.smithi039.stderr:+ local param=//journal/object_pool 2024-04-25T20:10:53.684 INFO:tasks.workunit.client.0.smithi039.stderr:+ local expected_val=rbd 2024-04-25T20:10:53.684 INFO:tasks.workunit.client.0.smithi039.stderr:++ rbd --format xml journal info --image testrbdimport47106 2024-04-25T20:10:53.684 INFO:tasks.workunit.client.0.smithi039.stderr:++ xmlstarlet sel -t -v //journal/object_pool 2024-04-25T20:10:53.723 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-04-25T20:10:53.722+0000 7f2fe31c0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31805) UID: 0 2024-04-25T20:10:53.724 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:53.724 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:53.724 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 0 2024-04-25T20:10:53.724 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:53.733 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-25T20:10:53.736 INFO:tasks.workunit.client.0.smithi039.stderr:+ local val=rbd 2024-04-25T20:10:53.736 INFO:tasks.workunit.client.0.smithi039.stderr:+ test rbd = rbd 2024-04-25T20:10:53.736 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd remove testrbdimport47106 2024-04-25T20:10:53.818 INFO:tasks.workunit.client.0.smithi039.stderr: Removing image: 1% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 12% complete... Removing image: 14% complete... Removing image: 15% complete...2024-04-25T20:10:53.817+0000 7f72705e4640 0 -- 172.21.15.39:0/1567854968 >> [v2:172.21.15.135:6808/31725,v1:172.21.15.135:6809/31725] conn(0x7f724c008be0 msgr2=0x7f724c00b0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-25T20:10:53.823 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:53.823+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:53.834 INFO:tasks.workunit.client.0.smithi039.stderr: Removing image: 17% complete... Removing image: 18% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 23% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 37% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 45% complete... 2024-04-25T20:10:53.833+0000 7f7272047640 0 -- 172.21.15.39:0/1567854968 >> [v2:172.21.15.135:6816/31737,v1:172.21.15.135:6817/31737] conn(0x5604ca7f5e20 msgr2=0x5604ca8236b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-25T20:10:53.848 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:53.848 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:53.848 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:53.848 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:53.848 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:53.859 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-25T20:10:53.883 INFO:tasks.workunit.client.0.smithi039.stderr:Removing image: 46% complete... Removing image: 48% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 62% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 73% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 87% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 98% complete... Removing image: 100% complete...done. 2024-04-25T20:10:53.886 INFO:tasks.workunit.client.0.smithi039.stderr:+ set +x 2024-04-25T20:10:53.886 INFO:tasks.workunit.client.0.smithi039.stderr:+ test_rbd_feature 2024-04-25T20:10:53.886 INFO:tasks.workunit.client.0.smithi039.stderr:+ local image=testrbdfeature47106 2024-04-25T20:10:53.886 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd create --image-feature exclusive-lock --size 256 testrbdfeature47106 2024-04-25T20:10:53.924 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-04-25T20:10:53.924+0000 7fecb1460640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31823) UID: 0 2024-04-25T20:10:53.933 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd feature enable testrbdfeature47106 journaling --journal-pool rbd --journal-object-size 20M --journal-splay-width 6 2024-04-25T20:10:53.995 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd_assert_eq testrbdfeature47106 'journal info' //journal/order 25 2024-04-25T20:10:53.995 INFO:tasks.workunit.client.0.smithi039.stderr:+ local image=testrbdfeature47106 2024-04-25T20:10:53.995 INFO:tasks.workunit.client.0.smithi039.stderr:+ local 'cmd=journal info' 2024-04-25T20:10:53.995 INFO:tasks.workunit.client.0.smithi039.stderr:+ local param=//journal/order 2024-04-25T20:10:53.995 INFO:tasks.workunit.client.0.smithi039.stderr:+ local expected_val=25 2024-04-25T20:10:53.995 INFO:tasks.workunit.client.0.smithi039.stderr:++ rbd --format xml journal info --image testrbdfeature47106 2024-04-25T20:10:53.995 INFO:tasks.workunit.client.0.smithi039.stderr:++ xmlstarlet sel -t -v //journal/order 2024-04-25T20:10:54.006 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:54.006 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:54.006 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:54.006 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [ 2024-04-25T20:10:54.006 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.006 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.5018.0:54 1.5 1:b4389475:::rbd_header.134bdebc9a96:head [watch unwatch cookie 140294301420976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-25T20:10:54.006 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:49.560026+0000", 2024-04-25T20:10:54.006 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 4.4450263210000003, 2024-04-25T20:10:54.007 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0051735660000000001, 2024-04-25T20:10:54.007 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.007 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:54.007 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:54.007 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.5018", 2024-04-25T20:10:54.007 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/712670978", 2024-04-25T20:10:54.007 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 54 2024-04-25T20:10:54.007 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.007 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.007 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.007 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.007 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:49.560026+0000", 2024-04-25T20:10:54.007 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.007 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.007 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.007 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.008 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:49.560026+0000", 2024-04-25T20:10:54.008 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.008 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.008 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.008 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.008 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:49.560026+0000", 2024-04-25T20:10:54.008 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:54.008 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.008 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.008 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.008 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:49.560028+0000", 2024-04-25T20:10:54.008 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.413e-06 2024-04-25T20:10:54.008 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.008 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.008 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.009 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:49.560038+0000", 2024-04-25T20:10:54.009 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.2979999999999997e-06 2024-04-25T20:10:54.009 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.009 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.009 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.009 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:49.560043+0000", 2024-04-25T20:10:54.009 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1059999999999999e-06 2024-04-25T20:10:54.009 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.009 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.009 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.009 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:49.560402+0000", 2024-04-25T20:10:54.009 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00035977300000000003 2024-04-25T20:10:54.009 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.009 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.010 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:54.010 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:49.560484+0000", 2024-04-25T20:10:54.010 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.1990000000000006e-05 2024-04-25T20:10:54.010 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.010 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.010 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for subops from 6", 2024-04-25T20:10:54.010 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:49.560582+0000", 2024-04-25T20:10:54.010 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.7527000000000003e-05 2024-04-25T20:10:54.010 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.010 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.010 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "sub_op_commit_rec", 2024-04-25T20:10:54.010 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:49.565026+0000", 2024-04-25T20:10:54.010 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0044444000000000003 2024-04-25T20:10:54.010 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.011 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.011 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "op_commit", 2024-04-25T20:10:54.011 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:49.565060+0000", 2024-04-25T20:10:54.011 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3325999999999999e-05 2024-04-25T20:10:54.011 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.011 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.011 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:54.011 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:49.565107+0000", 2024-04-25T20:10:54.011 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.7744000000000002e-05 2024-04-25T20:10:54.011 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.011 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.011 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.011 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:49.565200+0000", 2024-04-25T20:10:54.011 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.2380000000000007e-05 2024-04-25T20:10:54.011 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.012 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.012 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.012 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.012 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.012 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.5018.0:55 1.5 1:b4389475:::rbd_header.134bdebc9a96:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-25T20:10:54.012 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:49.560455+0000", 2024-04-25T20:10:54.012 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 4.4445974010000002, 2024-04-25T20:10:54.012 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048634819999999997, 2024-04-25T20:10:54.012 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.012 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "started", 2024-04-25T20:10:54.012 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:54.012 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.5018", 2024-04-25T20:10:54.012 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/712670978", 2024-04-25T20:10:54.012 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 55 2024-04-25T20:10:54.012 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.013 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.013 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.013 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.013 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:49.560455+0000", 2024-04-25T20:10:54.013 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.013 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.013 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.013 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.013 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:49.560455+0000", 2024-04-25T20:10:54.013 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.013 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.013 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.013 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.013 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:49.560454+0000", 2024-04-25T20:10:54.013 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.999999 2024-04-25T20:10:54.014 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.014 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.014 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.014 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:49.560459+0000", 2024-04-25T20:10:54.014 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.2570000000000001e-06 2024-04-25T20:10:54.014 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.014 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.014 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.014 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:49.560468+0000", 2024-04-25T20:10:54.014 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.0890000000000006e-06 2024-04-25T20:10:54.014 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.014 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.014 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.014 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:49.560473+0000", 2024-04-25T20:10:54.015 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.9069999999999998e-06 2024-04-25T20:10:54.015 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.015 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.015 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.015 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:49.560779+0000", 2024-04-25T20:10:54.015 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00030640600000000001 2024-04-25T20:10:54.015 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.015 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.015 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "waiting for rw locks", 2024-04-25T20:10:54.015 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:49.560823+0000", 2024-04-25T20:10:54.015 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.3526000000000003e-05 2024-04-25T20:10:54.015 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.015 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.015 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.015 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:49.565223+0000", 2024-04-25T20:10:54.016 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0044005329999999999 2024-04-25T20:10:54.016 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.016 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.016 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:54.016 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:49.565268+0000", 2024-04-25T20:10:54.016 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.5083000000000002e-05 2024-04-25T20:10:54.016 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.016 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.016 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.016 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:49.565319+0000", 2024-04-25T20:10:54.016 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.0444e-05 2024-04-25T20:10:54.016 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.016 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.016 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.017 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.017 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.017 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.5529.0:117 1.5 1:ba0692f3:::rbd_data.156c6e127736.000000000000002e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e62)", 2024-04-25T20:10:54.017 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:52.662903+0000", 2024-04-25T20:10:54.017 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 1.342149128, 2024-04-25T20:10:54.017 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0039245779999999997, 2024-04-25T20:10:54.017 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.017 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "started", 2024-04-25T20:10:54.017 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:54.017 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.5529", 2024-04-25T20:10:54.017 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/517788662", 2024-04-25T20:10:54.017 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 117 2024-04-25T20:10:54.017 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.017 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.017 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.018 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.018 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:52.662903+0000", 2024-04-25T20:10:54.018 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.018 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.018 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.018 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.018 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:52.662903+0000", 2024-04-25T20:10:54.018 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.018 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.018 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.018 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.018 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:52.662903+0000", 2024-04-25T20:10:54.018 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.999999 2024-04-25T20:10:54.018 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.019 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.019 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.019 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:52.662908+0000", 2024-04-25T20:10:54.019 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.4650000000000002e-06 2024-04-25T20:10:54.019 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.019 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.019 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.019 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:52.662920+0000", 2024-04-25T20:10:54.019 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.1637000000000001e-05 2024-04-25T20:10:54.019 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.019 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.019 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.019 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:52.662927+0000", 2024-04-25T20:10:54.019 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.6699999999999994e-06 2024-04-25T20:10:54.020 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.020 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.020 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.020 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:52.662992+0000", 2024-04-25T20:10:54.020 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.4919999999999995e-05 2024-04-25T20:10:54.020 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.020 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.020 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:54.020 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:52.663102+0000", 2024-04-25T20:10:54.020 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000109738 2024-04-25T20:10:54.020 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.020 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.020 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.020 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:52.666828+0000", 2024-04-25T20:10:54.021 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0037260420000000002 2024-04-25T20:10:54.021 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.021 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.021 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.021 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.021 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.021 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.5547.0:56 1.5 1:ae08bc43:::rbd_data.141861bceda4.0000000000000003:head [delete] snapc 6=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e63)", 2024-04-25T20:10:54.021 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.114114+0000", 2024-04-25T20:10:54.021 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.89093859600000003, 2024-04-25T20:10:54.021 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.005786167, 2024-04-25T20:10:54.021 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.021 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "started", 2024-04-25T20:10:54.021 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:54.021 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.5547", 2024-04-25T20:10:54.021 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3099364282", 2024-04-25T20:10:54.021 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 56 2024-04-25T20:10:54.022 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.022 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.022 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.022 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.022 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.114114+0000", 2024-04-25T20:10:54.022 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.022 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.022 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.022 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.022 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.114114+0000", 2024-04-25T20:10:54.022 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.022 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.022 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.022 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.023 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.114112+0000", 2024-04-25T20:10:54.023 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999981 2024-04-25T20:10:54.023 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.023 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.023 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.023 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.114119+0000", 2024-04-25T20:10:54.023 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.81e-06 2024-04-25T20:10:54.023 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.023 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.023 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.023 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.114133+0000", 2024-04-25T20:10:54.023 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.3818000000000001e-05 2024-04-25T20:10:54.023 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.023 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.023 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.023 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.114141+0000", 2024-04-25T20:10:54.024 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.9929999999999995e-06 2024-04-25T20:10:54.024 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.024 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.024 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.024 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.114195+0000", 2024-04-25T20:10:54.024 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.4175999999999998e-05 2024-04-25T20:10:54.024 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.024 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.024 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:54.024 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.114318+0000", 2024-04-25T20:10:54.024 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00012249799999999999 2024-04-25T20:10:54.024 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.024 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.024 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.024 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.119900+0000", 2024-04-25T20:10:54.024 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.005582559 2024-04-25T20:10:54.025 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.025 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.025 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.025 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.025 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.025 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.5547.0:73 1.5 1:ae4fc370:::rbd_data.141861bceda4.0000000000000014:head [delete] snapc 6=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e63)", 2024-04-25T20:10:54.025 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.119781+0000", 2024-04-25T20:10:54.025 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.88527184000000003, 2024-04-25T20:10:54.025 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0049891349999999996, 2024-04-25T20:10:54.025 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.025 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "started", 2024-04-25T20:10:54.025 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:54.025 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.5547", 2024-04-25T20:10:54.025 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/3099364282", 2024-04-25T20:10:54.025 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 73 2024-04-25T20:10:54.026 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.026 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.026 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.026 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.026 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.119781+0000", 2024-04-25T20:10:54.026 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.026 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.026 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.026 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.026 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.119781+0000", 2024-04-25T20:10:54.026 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.026 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.026 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.026 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.027 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.119780+0000", 2024-04-25T20:10:54.027 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.999999 2024-04-25T20:10:54.027 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.027 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.027 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.027 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.119785+0000", 2024-04-25T20:10:54.027 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1019999999999996e-06 2024-04-25T20:10:54.027 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.027 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.027 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.027 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.119797+0000", 2024-04-25T20:10:54.027 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.2184000000000001e-05 2024-04-25T20:10:54.027 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.027 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.028 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.028 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.119804+0000", 2024-04-25T20:10:54.028 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.126e-06 2024-04-25T20:10:54.028 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.028 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.028 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.028 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.119939+0000", 2024-04-25T20:10:54.028 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00013469600000000001 2024-04-25T20:10:54.028 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.028 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.028 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:54.028 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.120035+0000", 2024-04-25T20:10:54.028 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.5594999999999995e-05 2024-04-25T20:10:54.028 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.029 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.029 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.029 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.124770+0000", 2024-04-25T20:10:54.029 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0047352289999999997 2024-04-25T20:10:54.029 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.029 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.029 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.029 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.029 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.029 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "pg_update_log_missing(1.2 epoch 63/51 rep_tid 144 entries 63'94 (0'0) error 1:505a5d18:::rbd_data.141861bceda4.000000000000001c:head by client.5547.0:81 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 63'91)", 2024-04-25T20:10:54.029 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.120794+0000", 2024-04-25T20:10:54.029 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.88425866600000003, 2024-04-25T20:10:54.029 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0038659940000000002, 2024-04-25T20:10:54.029 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.029 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "reached pg", 2024-04-25T20:10:54.030 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.030 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.030 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.030 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.120794+0000", 2024-04-25T20:10:54.030 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.030 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.030 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.030 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.030 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.120794+0000", 2024-04-25T20:10:54.030 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.030 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.030 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.030 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.030 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.120793+0000", 2024-04-25T20:10:54.030 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.999999 2024-04-25T20:10:54.031 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.031 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.031 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.031 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.120802+0000", 2024-04-25T20:10:54.031 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.7520000000000002e-06 2024-04-25T20:10:54.031 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.031 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.031 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.031 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.120819+0000", 2024-04-25T20:10:54.031 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.7056999999999999e-05 2024-04-25T20:10:54.031 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.031 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.031 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.031 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.120830+0000", 2024-04-25T20:10:54.031 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.1248e-05 2024-04-25T20:10:54.032 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.032 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.032 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.032 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.120888+0000", 2024-04-25T20:10:54.032 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.7784000000000001e-05 2024-04-25T20:10:54.032 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.032 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.032 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.032 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.124660+0000", 2024-04-25T20:10:54.032 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0037719849999999998 2024-04-25T20:10:54.032 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.032 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.032 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.032 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.033 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.033 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.5607.0:73 1.5 1:a7ad6c49:::rbd_data.15c034b083aa.0000000000000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e64)", 2024-04-25T20:10:54.033 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.817619+0000", 2024-04-25T20:10:54.033 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.187433507, 2024-04-25T20:10:54.033 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0045161919999999996, 2024-04-25T20:10:54.033 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.033 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "started", 2024-04-25T20:10:54.033 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:54.033 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.5607", 2024-04-25T20:10:54.033 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/1567854968", 2024-04-25T20:10:54.033 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 73 2024-04-25T20:10:54.033 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.033 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.033 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.033 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.034 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.817619+0000", 2024-04-25T20:10:54.034 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.034 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.034 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.034 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.034 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.817619+0000", 2024-04-25T20:10:54.034 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.034 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.034 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.034 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.034 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.817617+0000", 2024-04-25T20:10:54.034 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999981 2024-04-25T20:10:54.034 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.034 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.035 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.035 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.817627+0000", 2024-04-25T20:10:54.035 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.3870000000000001e-06 2024-04-25T20:10:54.035 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.035 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.035 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.035 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.817640+0000", 2024-04-25T20:10:54.035 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.3811e-05 2024-04-25T20:10:54.035 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.035 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.035 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.035 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.817649+0000", 2024-04-25T20:10:54.035 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.6039999999999993e-06 2024-04-25T20:10:54.035 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.035 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.035 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.036 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.817708+0000", 2024-04-25T20:10:54.036 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.9095999999999998e-05 2024-04-25T20:10:54.036 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.036 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.036 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:54.036 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.817845+0000", 2024-04-25T20:10:54.036 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000137455 2024-04-25T20:10:54.036 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.036 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.036 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.036 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.822135+0000", 2024-04-25T20:10:54.036 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0042897949999999999 2024-04-25T20:10:54.036 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.036 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.037 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.037 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.037 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.037 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.5607.0:75 1.5 1:a83dd2c6:::rbd_data.15c034b083aa.0000000000000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e64)", 2024-04-25T20:10:54.037 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.817663+0000", 2024-04-25T20:10:54.037 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.18738929200000001, 2024-04-25T20:10:54.037 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0084998390000000004, 2024-04-25T20:10:54.037 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.037 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "started", 2024-04-25T20:10:54.037 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:54.037 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.5607", 2024-04-25T20:10:54.037 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/1567854968", 2024-04-25T20:10:54.037 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 75 2024-04-25T20:10:54.037 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.037 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.038 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.038 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.038 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.817663+0000", 2024-04-25T20:10:54.038 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.038 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.038 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.038 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.038 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.817663+0000", 2024-04-25T20:10:54.038 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.038 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.038 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.038 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.038 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.817663+0000", 2024-04-25T20:10:54.038 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:54.038 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.039 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.039 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.039 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.817666+0000", 2024-04-25T20:10:54.039 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3610000000000001e-06 2024-04-25T20:10:54.039 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.039 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.039 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.039 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.817673+0000", 2024-04-25T20:10:54.039 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0260000000000002e-06 2024-04-25T20:10:54.039 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.039 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.039 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.039 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.817678+0000", 2024-04-25T20:10:54.039 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.074e-06 2024-04-25T20:10:54.039 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.040 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.040 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.040 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.818167+0000", 2024-04-25T20:10:54.040 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00048829500000000003 2024-04-25T20:10:54.040 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.040 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.040 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:54.040 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.818375+0000", 2024-04-25T20:10:54.040 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00020827700000000001 2024-04-25T20:10:54.040 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.040 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.040 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.040 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.826163+0000", 2024-04-25T20:10:54.040 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0077883559999999998 2024-04-25T20:10:54.041 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.041 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.041 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.041 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.041 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.041 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.5607.0:79 1.5 1:b407d648:::rbd_data.15c034b083aa.0000000000000008:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e64)", 2024-04-25T20:10:54.041 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.817695+0000", 2024-04-25T20:10:54.041 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.187357405, 2024-04-25T20:10:54.041 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0085855889999999994, 2024-04-25T20:10:54.041 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.041 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "started", 2024-04-25T20:10:54.041 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:54.041 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.5607", 2024-04-25T20:10:54.041 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/1567854968", 2024-04-25T20:10:54.041 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 79 2024-04-25T20:10:54.042 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.042 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.042 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.042 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.042 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.817695+0000", 2024-04-25T20:10:54.042 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.042 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.042 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.042 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.042 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.817695+0000", 2024-04-25T20:10:54.042 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.042 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.042 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.043 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.043 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.817695+0000", 2024-04-25T20:10:54.043 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:54.043 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.043 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.043 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.043 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.817698+0000", 2024-04-25T20:10:54.043 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.033e-06 2024-04-25T20:10:54.043 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.043 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.043 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.043 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.817705+0000", 2024-04-25T20:10:54.043 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0960000000000003e-06 2024-04-25T20:10:54.044 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.044 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.044 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.044 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.817713+0000", 2024-04-25T20:10:54.044 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.9680000000000006e-06 2024-04-25T20:10:54.044 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.044 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.044 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.044 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.818773+0000", 2024-04-25T20:10:54.044 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00106047 2024-04-25T20:10:54.044 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.044 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.044 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:54.044 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.818960+0000", 2024-04-25T20:10:54.044 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000186365 2024-04-25T20:10:54.045 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.045 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.045 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.045 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.826281+0000", 2024-04-25T20:10:54.045 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0073210580000000001 2024-04-25T20:10:54.045 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.045 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.045 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.045 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.045 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.045 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "pg_update_log_missing(1.2 epoch 64/51 rep_tid 160 entries 64'103 (0'0) error 1:5f3b6013:::rbd_data.15c034b083aa.000000000000000d:head by client.5607.0:85 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 63'101)", 2024-04-25T20:10:54.045 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.819150+0000", 2024-04-25T20:10:54.045 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.185902332, 2024-04-25T20:10:54.045 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0065715299999999999, 2024-04-25T20:10:54.046 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.046 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "reached pg", 2024-04-25T20:10:54.046 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.046 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.046 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.046 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.819150+0000", 2024-04-25T20:10:54.046 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.046 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.046 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.046 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.046 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.819150+0000", 2024-04-25T20:10:54.046 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.046 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.046 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.047 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.047 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.819149+0000", 2024-04-25T20:10:54.047 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.999999 2024-04-25T20:10:54.047 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.047 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.047 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.047 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.819158+0000", 2024-04-25T20:10:54.047 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.3440000000000001e-06 2024-04-25T20:10:54.047 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.047 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.047 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.047 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.819174+0000", 2024-04-25T20:10:54.047 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.6197000000000001e-05 2024-04-25T20:10:54.047 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.047 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.048 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.048 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.819185+0000", 2024-04-25T20:10:54.048 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0746e-05 2024-04-25T20:10:54.048 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.048 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.048 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.048 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.819277+0000", 2024-04-25T20:10:54.048 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.2378000000000002e-05 2024-04-25T20:10:54.048 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.048 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.048 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.048 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.825722+0000", 2024-04-25T20:10:54.048 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0064446950000000003 2024-04-25T20:10:54.048 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.048 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.049 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.049 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.049 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.049 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.5607.0:89 1.5 1:a6fcfd4e:::rbd_data.15c034b083aa.0000000000000011:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e64)", 2024-04-25T20:10:54.049 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.822231+0000", 2024-04-25T20:10:54.049 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.18282131900000001, 2024-04-25T20:10:54.049 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0079359129999999993, 2024-04-25T20:10:54.049 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.049 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "started", 2024-04-25T20:10:54.049 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:54.049 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.5607", 2024-04-25T20:10:54.049 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/1567854968", 2024-04-25T20:10:54.049 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 89 2024-04-25T20:10:54.050 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.050 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.050 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.050 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.050 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.822231+0000", 2024-04-25T20:10:54.050 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.050 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.050 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.050 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.050 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.822231+0000", 2024-04-25T20:10:54.050 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.050 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.050 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.050 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.050 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.822231+0000", 2024-04-25T20:10:54.051 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.999999 2024-04-25T20:10:54.051 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.051 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.051 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.051 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.822234+0000", 2024-04-25T20:10:54.051 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.873e-06 2024-04-25T20:10:54.051 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.051 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.051 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.051 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.822243+0000", 2024-04-25T20:10:54.051 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.9360000000000005e-06 2024-04-25T20:10:54.051 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.051 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.051 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.051 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.822248+0000", 2024-04-25T20:10:54.051 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.6580000000000001e-06 2024-04-25T20:10:54.052 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.052 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.052 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.052 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.822281+0000", 2024-04-25T20:10:54.052 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3028999999999998e-05 2024-04-25T20:10:54.052 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.052 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.052 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:54.052 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.822399+0000", 2024-04-25T20:10:54.052 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000117885 2024-04-25T20:10:54.052 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.052 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.052 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.052 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.830167+0000", 2024-04-25T20:10:54.053 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0077682949999999997 2024-04-25T20:10:54.053 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.053 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.053 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.053 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.053 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.053 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "pg_update_log_missing(1.2 epoch 64/51 rep_tid 161 entries 64'104 (0'0) error 1:566f4a86:::rbd_data.15c034b083aa.0000000000000010:head by client.5607.0:88 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 63'101)", 2024-04-25T20:10:54.053 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.822396+0000", 2024-04-25T20:10:54.053 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.18265630999999999, 2024-04-25T20:10:54.053 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0074526139999999998, 2024-04-25T20:10:54.053 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.053 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "reached pg", 2024-04-25T20:10:54.053 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.053 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.053 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.054 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.822396+0000", 2024-04-25T20:10:54.054 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.054 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.054 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.054 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.054 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.822396+0000", 2024-04-25T20:10:54.054 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.054 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.054 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.054 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.054 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.822396+0000", 2024-04-25T20:10:54.054 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:54.054 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.054 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.055 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.055 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.822401+0000", 2024-04-25T20:10:54.055 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1680000000000003e-06 2024-04-25T20:10:54.055 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.055 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.055 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.055 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.822414+0000", 2024-04-25T20:10:54.055 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.2712e-05 2024-04-25T20:10:54.055 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.055 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.055 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.055 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.822421+0000", 2024-04-25T20:10:54.055 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.8460000000000002e-06 2024-04-25T20:10:54.055 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.056 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.056 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.056 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.822455+0000", 2024-04-25T20:10:54.056 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.4332000000000003e-05 2024-04-25T20:10:54.056 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.056 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.056 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.056 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.829849+0000", 2024-04-25T20:10:54.056 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0073938950000000002 2024-04-25T20:10:54.056 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.056 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.056 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.056 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.056 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.056 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.5607.0:92 1.5 1:b6be0fa9:::rbd_data.15c034b083aa.0000000000000014:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e64)", 2024-04-25T20:10:54.056 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.826151+0000", 2024-04-25T20:10:54.057 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.17890133599999999, 2024-04-25T20:10:54.057 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0079687869999999997, 2024-04-25T20:10:54.057 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.057 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "started", 2024-04-25T20:10:54.057 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:54.057 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.5607", 2024-04-25T20:10:54.057 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/1567854968", 2024-04-25T20:10:54.057 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 92 2024-04-25T20:10:54.057 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.057 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.057 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.057 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.057 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.826151+0000", 2024-04-25T20:10:54.057 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.057 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.058 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.058 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.058 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.826151+0000", 2024-04-25T20:10:54.058 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.058 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.058 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.058 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.058 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.826150+0000", 2024-04-25T20:10:54.058 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.999999 2024-04-25T20:10:54.058 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.058 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.058 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.058 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.826155+0000", 2024-04-25T20:10:54.058 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.7840000000000003e-06 2024-04-25T20:10:54.059 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.059 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.059 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.059 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.826163+0000", 2024-04-25T20:10:54.059 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.8639999999999996e-06 2024-04-25T20:10:54.059 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.059 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.059 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.059 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.826169+0000", 2024-04-25T20:10:54.059 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.8470000000000001e-06 2024-04-25T20:10:54.059 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.059 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.059 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.059 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.826312+0000", 2024-04-25T20:10:54.060 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000142898 2024-04-25T20:10:54.060 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.060 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.060 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:54.060 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.826384+0000", 2024-04-25T20:10:54.060 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2581000000000002e-05 2024-04-25T20:10:54.060 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.060 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.060 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.060 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.834120+0000", 2024-04-25T20:10:54.060 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0077356170000000002 2024-04-25T20:10:54.060 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.060 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.060 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.060 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.061 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.061 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.5607.0:100 1.5 1:ad7f2853:::rbd_data.15c034b083aa.000000000000001c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e64)", 2024-04-25T20:10:54.061 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.830278+0000", 2024-04-25T20:10:54.061 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.174774877, 2024-04-25T20:10:54.061 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0075867119999999998, 2024-04-25T20:10:54.061 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.061 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "started", 2024-04-25T20:10:54.061 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:54.061 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.5607", 2024-04-25T20:10:54.061 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/1567854968", 2024-04-25T20:10:54.061 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 100 2024-04-25T20:10:54.061 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.061 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.061 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.061 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.062 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.830278+0000", 2024-04-25T20:10:54.062 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.062 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.062 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.062 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.062 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.830278+0000", 2024-04-25T20:10:54.062 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.062 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.062 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.062 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.062 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.830277+0000", 2024-04-25T20:10:54.062 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.999999 2024-04-25T20:10:54.062 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.063 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.063 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.063 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.830283+0000", 2024-04-25T20:10:54.063 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.4219999999999997e-06 2024-04-25T20:10:54.063 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.063 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.063 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.063 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.830293+0000", 2024-04-25T20:10:54.063 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.4350000000000003e-06 2024-04-25T20:10:54.063 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.063 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.063 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.063 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.830297+0000", 2024-04-25T20:10:54.063 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.617e-06 2024-04-25T20:10:54.064 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.064 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.064 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.064 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.830329+0000", 2024-04-25T20:10:54.064 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.1729000000000001e-05 2024-04-25T20:10:54.064 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.064 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.064 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:54.064 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.830451+0000", 2024-04-25T20:10:54.064 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000122235 2024-04-25T20:10:54.064 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.064 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.064 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.064 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.837864+0000", 2024-04-25T20:10:54.064 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.007413233 2024-04-25T20:10:54.065 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.065 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.065 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.065 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.065 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.065 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "pg_update_log_missing(1.2 epoch 64/51 rep_tid 163 entries 64'105 (0'0) error 1:5dd2ee7c:::rbd_data.15c034b083aa.000000000000001e:head by client.5607.0:102 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 64'103)", 2024-04-25T20:10:54.065 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.834776+0000", 2024-04-25T20:10:54.065 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.17027671699999999, 2024-04-25T20:10:54.065 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0064690540000000001, 2024-04-25T20:10:54.065 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.065 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "reached pg", 2024-04-25T20:10:54.065 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.065 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.065 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.065 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.834776+0000", 2024-04-25T20:10:54.066 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.066 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.066 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.066 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.066 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.834776+0000", 2024-04-25T20:10:54.066 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.066 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.066 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.066 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.066 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.834775+0000", 2024-04-25T20:10:54.066 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.999999 2024-04-25T20:10:54.066 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.066 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.066 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.067 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.834784+0000", 2024-04-25T20:10:54.067 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.1549999999999996e-06 2024-04-25T20:10:54.067 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.067 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.067 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.067 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.834801+0000", 2024-04-25T20:10:54.067 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.6427999999999999e-05 2024-04-25T20:10:54.067 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.067 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.067 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.067 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.834812+0000", 2024-04-25T20:10:54.067 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0903999999999999e-05 2024-04-25T20:10:54.067 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.067 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.068 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.068 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.834866+0000", 2024-04-25T20:10:54.068 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.4326000000000001e-05 2024-04-25T20:10:54.068 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.068 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.068 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.068 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.841245+0000", 2024-04-25T20:10:54.068 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0063789759999999997 2024-04-25T20:10:54.068 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.068 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.068 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.068 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.068 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.068 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.5607.0:110 1.5 1:a94b52a3:::rbd_data.15c034b083aa.0000000000000026:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e64)", 2024-04-25T20:10:54.068 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.838072+0000", 2024-04-25T20:10:54.069 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.16698028000000001, 2024-04-25T20:10:54.069 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0042811739999999996, 2024-04-25T20:10:54.069 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.069 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "started", 2024-04-25T20:10:54.069 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:54.069 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.5607", 2024-04-25T20:10:54.069 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/1567854968", 2024-04-25T20:10:54.069 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 110 2024-04-25T20:10:54.069 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.069 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.069 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.069 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.069 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.838072+0000", 2024-04-25T20:10:54.069 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.069 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.070 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.070 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.070 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.838072+0000", 2024-04-25T20:10:54.070 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.070 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.070 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.070 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.070 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.838071+0000", 2024-04-25T20:10:54.070 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.999999 2024-04-25T20:10:54.070 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.070 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.070 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.070 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.838078+0000", 2024-04-25T20:10:54.070 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.3980000000000004e-06 2024-04-25T20:10:54.071 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.071 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.071 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.071 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.838090+0000", 2024-04-25T20:10:54.071 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.2768999999999999e-05 2024-04-25T20:10:54.071 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.071 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.071 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.071 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.838098+0000", 2024-04-25T20:10:54.071 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2219999999999996e-06 2024-04-25T20:10:54.071 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.071 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.071 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.071 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.838131+0000", 2024-04-25T20:10:54.072 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.3436000000000003e-05 2024-04-25T20:10:54.072 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.072 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.072 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:54.072 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.838256+0000", 2024-04-25T20:10:54.072 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00012544699999999999 2024-04-25T20:10:54.072 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.072 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.072 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.072 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.842353+0000", 2024-04-25T20:10:54.072 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0040970920000000001 2024-04-25T20:10:54.072 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.072 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.072 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.073 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.073 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.073 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.5607.0:111 1.5 1:a4e30a95:::rbd_data.15c034b083aa.0000000000000027:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e64)", 2024-04-25T20:10:54.073 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.841744+0000", 2024-04-25T20:10:54.073 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.16330858100000001, 2024-04-25T20:10:54.073 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0049613610000000001, 2024-04-25T20:10:54.073 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.073 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "started", 2024-04-25T20:10:54.073 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:54.073 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.5607", 2024-04-25T20:10:54.073 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/1567854968", 2024-04-25T20:10:54.073 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 111 2024-04-25T20:10:54.073 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.073 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.074 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.074 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.074 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.841744+0000", 2024-04-25T20:10:54.074 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.074 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.074 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.074 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.074 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.841744+0000", 2024-04-25T20:10:54.074 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.074 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.074 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.074 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.074 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.841743+0000", 2024-04-25T20:10:54.074 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.999999 2024-04-25T20:10:54.074 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.075 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.075 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.075 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.841753+0000", 2024-04-25T20:10:54.075 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0069e-05 2024-04-25T20:10:54.075 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.075 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.075 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.075 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.841768+0000", 2024-04-25T20:10:54.075 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.4511999999999999e-05 2024-04-25T20:10:54.075 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.075 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.075 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.075 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.841775+0000", 2024-04-25T20:10:54.075 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0609999999999998e-06 2024-04-25T20:10:54.075 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.076 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.076 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.076 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.841821+0000", 2024-04-25T20:10:54.076 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.6788e-05 2024-04-25T20:10:54.076 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.076 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.076 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:54.076 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.841971+0000", 2024-04-25T20:10:54.076 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00014977299999999999 2024-04-25T20:10:54.076 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.076 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.076 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.076 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.846705+0000", 2024-04-25T20:10:54.076 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0047341420000000002 2024-04-25T20:10:54.076 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.076 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.077 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.077 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.077 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.077 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.5607.0:117 1.5 1:a8f48618:::rbd_data.15c034b083aa.000000000000002d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e64)", 2024-04-25T20:10:54.077 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.841966+0000", 2024-04-25T20:10:54.077 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.16308693199999999, 2024-04-25T20:10:54.077 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0075651970000000001, 2024-04-25T20:10:54.077 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.077 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "started", 2024-04-25T20:10:54.077 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:54.077 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.5607", 2024-04-25T20:10:54.077 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/1567854968", 2024-04-25T20:10:54.077 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 117 2024-04-25T20:10:54.077 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.077 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.077 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.078 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.078 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.841966+0000", 2024-04-25T20:10:54.078 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.078 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.078 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.078 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.078 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.841966+0000", 2024-04-25T20:10:54.078 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.078 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.078 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.078 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.078 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.841965+0000", 2024-04-25T20:10:54.078 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.999999 2024-04-25T20:10:54.078 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.079 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.079 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.079 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.841969+0000", 2024-04-25T20:10:54.079 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.3780000000000003e-06 2024-04-25T20:10:54.079 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.079 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.079 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.079 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.841979+0000", 2024-04-25T20:10:54.079 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0129000000000001e-05 2024-04-25T20:10:54.079 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.079 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.079 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.079 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.841986+0000", 2024-04-25T20:10:54.079 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.3470000000000004e-06 2024-04-25T20:10:54.079 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.080 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.080 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.080 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.842421+0000", 2024-04-25T20:10:54.080 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00043477400000000002 2024-04-25T20:10:54.080 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.080 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.080 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:54.080 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.842839+0000", 2024-04-25T20:10:54.080 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.000418849 2024-04-25T20:10:54.080 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.080 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.080 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.080 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.849531+0000", 2024-04-25T20:10:54.080 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0066915070000000002 2024-04-25T20:10:54.080 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.080 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.081 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.081 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.081 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.081 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "pg_update_log_missing(1.2 epoch 64/51 rep_tid 167 entries 64'106 (0'0) error 1:48b8c773:::rbd_data.15c034b083aa.000000000000002e:head by client.5607.0:118 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 64'104)", 2024-04-25T20:10:54.081 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.842558+0000", 2024-04-25T20:10:54.081 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.16249455800000001, 2024-04-25T20:10:54.081 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0068173399999999999, 2024-04-25T20:10:54.081 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.081 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "reached pg", 2024-04-25T20:10:54.081 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.081 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.081 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.081 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.842558+0000", 2024-04-25T20:10:54.081 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.081 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.082 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.082 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.082 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.842558+0000", 2024-04-25T20:10:54.082 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.082 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.082 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.082 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.082 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.842557+0000", 2024-04-25T20:10:54.082 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:54.082 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.082 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.082 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.082 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.842568+0000", 2024-04-25T20:10:54.083 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0781999999999999e-05 2024-04-25T20:10:54.083 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.083 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.083 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.083 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.842585+0000", 2024-04-25T20:10:54.083 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.6396e-05 2024-04-25T20:10:54.083 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.083 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.083 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.083 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.842596+0000", 2024-04-25T20:10:54.083 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.168e-05 2024-04-25T20:10:54.083 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.083 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.083 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.083 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.842668+0000", 2024-04-25T20:10:54.084 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.1767000000000006e-05 2024-04-25T20:10:54.084 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.084 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.084 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.084 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.849375+0000", 2024-04-25T20:10:54.084 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0067073560000000003 2024-04-25T20:10:54.084 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.084 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.084 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.084 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.084 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.084 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_op(client.5607.0:133 1.5 1:baf8245a:::rbd_data.15c034b083aa.000000000000003d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e64)", 2024-04-25T20:10:54.084 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.853046+0000", 2024-04-25T20:10:54.084 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.152006472, 2024-04-25T20:10:54.084 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0046400219999999997, 2024-04-25T20:10:54.085 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.085 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "started", 2024-04-25T20:10:54.085 INFO:teuthology.orchestra.run.smithi135.stdout: "client_info": { 2024-04-25T20:10:54.085 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.5607", 2024-04-25T20:10:54.085 INFO:teuthology.orchestra.run.smithi135.stdout: "client_addr": "172.21.15.39:0/1567854968", 2024-04-25T20:10:54.085 INFO:teuthology.orchestra.run.smithi135.stdout: "tid": 133 2024-04-25T20:10:54.085 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.085 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.085 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.085 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.085 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.853046+0000", 2024-04-25T20:10:54.085 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.085 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.085 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.085 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.086 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.853046+0000", 2024-04-25T20:10:54.086 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.086 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.086 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.086 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.086 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.853044+0000", 2024-04-25T20:10:54.086 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999981 2024-04-25T20:10:54.086 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.086 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.086 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.086 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.853052+0000", 2024-04-25T20:10:54.086 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.6499999999999996e-06 2024-04-25T20:10:54.086 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.086 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.087 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.087 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.853068+0000", 2024-04-25T20:10:54.087 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.628e-05 2024-04-25T20:10:54.087 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.087 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.087 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.087 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.853076+0000", 2024-04-25T20:10:54.087 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.1019999999999997e-06 2024-04-25T20:10:54.087 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.087 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.087 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.087 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.853136+0000", 2024-04-25T20:10:54.087 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.9642999999999998e-05 2024-04-25T20:10:54.087 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.087 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.087 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:54.088 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.853272+0000", 2024-04-25T20:10:54.088 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00013601299999999999 2024-04-25T20:10:54.088 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.088 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.088 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.088 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.857686+0000", 2024-04-25T20:10:54.088 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0044141909999999996 2024-04-25T20:10:54.088 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.088 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.088 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.088 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.088 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.088 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:54.089 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-25T20:10:54.091 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-25T20:10:54.091 INFO:tasks.ceph.osd.6.smithi135.stderr:2024-04-25T20:10:54.090+0000 7fa97773e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31721) UID: 0 2024-04-25T20:10:54.092 INFO:tasks.workunit.client.0.smithi039.stderr:+ local val=25 2024-04-25T20:10:54.092 INFO:tasks.workunit.client.0.smithi039.stderr:+ test 25 = 25 2024-04-25T20:10:54.092 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd_assert_eq testrbdfeature47106 'journal info' //journal/splay_width 6 2024-04-25T20:10:54.092 INFO:tasks.workunit.client.0.smithi039.stderr:+ local image=testrbdfeature47106 2024-04-25T20:10:54.092 INFO:tasks.workunit.client.0.smithi039.stderr:+ local 'cmd=journal info' 2024-04-25T20:10:54.092 INFO:tasks.workunit.client.0.smithi039.stderr:+ local param=//journal/splay_width 2024-04-25T20:10:54.092 INFO:tasks.workunit.client.0.smithi039.stderr:+ local expected_val=6 2024-04-25T20:10:54.092 INFO:tasks.workunit.client.0.smithi039.stderr:++ rbd --format xml journal info --image testrbdfeature47106 2024-04-25T20:10:54.092 INFO:tasks.workunit.client.0.smithi039.stderr:++ xmlstarlet sel -t -v //journal/splay_width 2024-04-25T20:10:54.103 INFO:tasks.workunit.client.0.smithi039.stderr:+ local val=6 2024-04-25T20:10:54.103 INFO:tasks.workunit.client.0.smithi039.stderr:+ test 6 = 6 2024-04-25T20:10:54.103 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd_assert_eq testrbdfeature47106 'journal info' //journal/object_pool rbd 2024-04-25T20:10:54.103 INFO:tasks.workunit.client.0.smithi039.stderr:+ local image=testrbdfeature47106 2024-04-25T20:10:54.103 INFO:tasks.workunit.client.0.smithi039.stderr:+ local 'cmd=journal info' 2024-04-25T20:10:54.103 INFO:tasks.workunit.client.0.smithi039.stderr:+ local param=//journal/object_pool 2024-04-25T20:10:54.103 INFO:tasks.workunit.client.0.smithi039.stderr:+ local expected_val=rbd 2024-04-25T20:10:54.104 INFO:tasks.workunit.client.0.smithi039.stderr:++ rbd --format xml journal info --image testrbdfeature47106 2024-04-25T20:10:54.104 INFO:tasks.workunit.client.0.smithi039.stderr:++ xmlstarlet sel -t -v //journal/object_pool 2024-04-25T20:10:54.158 INFO:tasks.workunit.client.0.smithi039.stderr:+ local val=rbd 2024-04-25T20:10:54.158 INFO:tasks.workunit.client.0.smithi039.stderr:+ test rbd = rbd 2024-04-25T20:10:54.159 INFO:tasks.workunit.client.0.smithi039.stderr:+ rbd remove testrbdfeature47106 2024-04-25T20:10:54.191 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:54.191+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:54.235 INFO:tasks.workunit.client.0.smithi039.stderr: Removing image: 1% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 12% complete... Removing image: 14% complete...2024-04-25T20:10:54.234+0000 7f6e1af52640 0 -- 172.21.15.39:0/2099992029 >> [v2:172.21.15.39:6816/31832,v1:172.21.15.39:6818/31832] conn(0x563ed1422820 msgr2=0x563ed1424cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-25T20:10:54.239 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:54.239 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [ 2024-04-25T20:10:54.239 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.239 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "pg_update_log_missing(1.1 epoch 64/21 rep_tid 178 entries 64'66 (0'0) error 1:935dc6bd:::rbd_data.14b1e20d27b8.0000000000000012:head by client.5661.0:89 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 64'64)", 2024-04-25T20:10:54.240 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:54.239448+0000", 2024-04-25T20:10:54.240 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.000386554, 2024-04-25T20:10:54.240 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.00040625200000000001, 2024-04-25T20:10:54.240 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.240 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "reached pg", 2024-04-25T20:10:54.240 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.240 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.240 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.240 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:54.239448+0000", 2024-04-25T20:10:54.240 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.240 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.240 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.240 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.240 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:54.239448+0000", 2024-04-25T20:10:54.241 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.241 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.241 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.241 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.241 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:54.239447+0000", 2024-04-25T20:10:54.241 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.999999 2024-04-25T20:10:54.241 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.241 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.241 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.241 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:54.239456+0000", 2024-04-25T20:10:54.241 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.4670000000000004e-06 2024-04-25T20:10:54.241 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.241 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.241 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.241 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:54.239473+0000", 2024-04-25T20:10:54.241 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.7343999999999999e-05 2024-04-25T20:10:54.241 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.242 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.242 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.242 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:54.239485+0000", 2024-04-25T20:10:54.242 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.1501000000000001e-05 2024-04-25T20:10:54.242 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.242 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.242 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.242 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:54.239527+0000", 2024-04-25T20:10:54.242 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.2573e-05 2024-04-25T20:10:54.242 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.242 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.242 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.242 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.242 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-25T20:10:54.243 INFO:teuthology.orchestra.run.smithi135.stdout: "num_ops": 1 2024-04-25T20:10:54.243 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:54.246 INFO:tasks.workunit.client.0.smithi039.stderr: Removing image: 15% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 23% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 37% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 48% complete...2024-04-25T20:10:54.240+0000 7f6e1c1b4640 0 -- 172.21.15.39:0/2099992029 >> [v2:172.21.15.135:6816/31737,v1:172.21.15.135:6817/31737] conn(0x563ed13d2740 msgr2=0x563ed13d4b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-25T20:10:54.251 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-25T20:10:54.273 INFO:tasks.workunit.client.0.smithi039.stderr: Removing image: 50% complete... Removing image: 51% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 62% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 73% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 87% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 98% complete... Removing image: 100% complete...done. 2024-04-25T20:10:54.276 INFO:tasks.workunit.client.0.smithi039.stderr:+ set +x 2024-04-25T20:10:54.276 INFO:tasks.workunit.client.0.smithi039.stdout:OK 2024-04-25T20:10:54.291 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-04-25T20:10:54.291+0000 7ffa587c7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31815) UID: 0 2024-04-25T20:10:54.307 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T20:10:54.307 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T20:10:54.372 INFO:tasks.workunit:Stopping ['rbd/journal.sh'] on client.0... 2024-04-25T20:10:54.372 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T20:10:54.392 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:54.392 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [], 2024-04-25T20:10:54.392 INFO:teuthology.orchestra.run.smithi135.stdout: "complaint_time": 30, 2024-04-25T20:10:54.392 INFO:teuthology.orchestra.run.smithi135.stdout: "num_blocked_ops": 0 2024-04-25T20:10:54.393 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:54.401 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-25T20:10:54.413 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:54.413+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:54.447 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-25T20:10:54.493 INFO:tasks.ceph.osd.5.smithi135.stderr:2024-04-25T20:10:54.493+0000 7ffac06b9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31716) UID: 0 2024-04-25T20:10:54.504 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-25T20:10:54.504 INFO:teuthology.orchestra.run.smithi135.stdout: "size": 20, 2024-04-25T20:10:54.504 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 600, 2024-04-25T20:10:54.504 INFO:teuthology.orchestra.run.smithi135.stdout: "ops": [ 2024-04-25T20:10:54.504 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.504 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "osd_repop(client.4982.0:14 1.4 e59/25 1:3d83a8fb:::journal_data.1.145ee3bcd187.3:head v 59'22, mlcod=59'17)", 2024-04-25T20:10:54.504 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:48.402359+0000", 2024-04-25T20:10:54.504 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 6.1007609829999998, 2024-04-25T20:10:54.504 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0045293260000000002, 2024-04-25T20:10:54.504 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.504 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-25T20:10:54.504 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.505 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.505 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.505 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:48.402359+0000", 2024-04-25T20:10:54.505 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.505 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.505 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.505 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.505 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:48.402359+0000", 2024-04-25T20:10:54.505 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.505 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.505 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.505 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.505 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:48.402359+0000", 2024-04-25T20:10:54.505 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:54.506 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.506 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.506 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.506 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:48.402371+0000", 2024-04-25T20:10:54.506 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.2352e-05 2024-04-25T20:10:54.506 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.506 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.506 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.506 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:48.402381+0000", 2024-04-25T20:10:54.506 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.1679999999999993e-06 2024-04-25T20:10:54.506 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.506 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.506 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.506 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:48.402386+0000", 2024-04-25T20:10:54.507 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.8109999999999999e-06 2024-04-25T20:10:54.507 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.507 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.507 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.507 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:48.402425+0000", 2024-04-25T20:10:54.507 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.8577e-05 2024-04-25T20:10:54.507 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.507 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.507 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "started", 2024-04-25T20:10:54.507 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:48.402440+0000", 2024-04-25T20:10:54.507 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.5e-05 2024-04-25T20:10:54.507 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.507 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.507 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "commit_sent", 2024-04-25T20:10:54.508 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:48.406848+0000", 2024-04-25T20:10:54.508 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0044080619999999999 2024-04-25T20:10:54.508 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.508 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.508 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.508 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:48.406889+0000", 2024-04-25T20:10:54.508 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.0828999999999998e-05 2024-04-25T20:10:54.508 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.508 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.508 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.508 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.508 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.508 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "pg_update_log_missing(1.1 epoch 61/21 rep_tid 24 entries 61'12 (0'0) error 1:9d721976:::rbd_data.145ee3bcd187.0000000000000013:head by client.5036.0:90 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 60'10)", 2024-04-25T20:10:54.508 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:50.092742+0000", 2024-04-25T20:10:54.508 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 4.4103789500000001, 2024-04-25T20:10:54.509 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0043816200000000001, 2024-04-25T20:10:54.509 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.509 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "reached pg", 2024-04-25T20:10:54.509 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.509 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.509 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.509 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:50.092742+0000", 2024-04-25T20:10:54.509 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.509 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.509 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.509 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.509 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:50.092742+0000", 2024-04-25T20:10:54.509 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.509 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.510 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.510 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.510 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:50.092741+0000", 2024-04-25T20:10:54.510 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.999999 2024-04-25T20:10:54.510 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.510 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.510 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.510 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:50.092752+0000", 2024-04-25T20:10:54.510 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.164e-05 2024-04-25T20:10:54.510 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.510 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.510 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.510 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:50.092767+0000", 2024-04-25T20:10:54.510 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.5003e-05 2024-04-25T20:10:54.511 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.511 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.511 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.511 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:50.092778+0000", 2024-04-25T20:10:54.511 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0675e-05 2024-04-25T20:10:54.511 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.511 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.511 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.511 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:50.092835+0000", 2024-04-25T20:10:54.511 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.6679999999999999e-05 2024-04-25T20:10:54.511 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.511 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.511 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.511 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:50.097123+0000", 2024-04-25T20:10:54.511 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0042885459999999999 2024-04-25T20:10:54.511 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.512 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.512 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.512 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.512 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.512 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "pg_update_log_missing(1.4 epoch 61/25 rep_tid 196 entries 61'32 (0'0) error 1:2689012b:::rbd_data.145ee3bcd187.0000000000000015:head by client.5036.0:92 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 61'29)", 2024-04-25T20:10:54.512 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:50.093288+0000", 2024-04-25T20:10:54.512 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 4.4098328850000001, 2024-04-25T20:10:54.512 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0038597470000000002, 2024-04-25T20:10:54.512 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.512 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "reached pg", 2024-04-25T20:10:54.512 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.512 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.512 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.512 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:50.093288+0000", 2024-04-25T20:10:54.512 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.513 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.513 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.513 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.513 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:50.093288+0000", 2024-04-25T20:10:54.513 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.513 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.513 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.513 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.513 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:50.093287+0000", 2024-04-25T20:10:54.513 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.999999 2024-04-25T20:10:54.513 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.513 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.513 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.513 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:50.093296+0000", 2024-04-25T20:10:54.513 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.4550000000000001e-06 2024-04-25T20:10:54.514 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.514 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.514 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.514 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:50.093315+0000", 2024-04-25T20:10:54.514 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.842e-05 2024-04-25T20:10:54.514 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.514 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.514 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.514 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:50.093325+0000", 2024-04-25T20:10:54.514 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0316000000000001e-05 2024-04-25T20:10:54.514 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.514 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.514 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.514 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:50.093386+0000", 2024-04-25T20:10:54.514 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.1418e-05 2024-04-25T20:10:54.515 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.515 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.515 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.515 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:50.097147+0000", 2024-04-25T20:10:54.515 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0037609190000000002 2024-04-25T20:10:54.515 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.515 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.515 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.515 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.515 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.515 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "pg_update_log_missing(1.4 epoch 62/25 rep_tid 300 entries 62'78 (0'0) error 1:23b662fa:::rbd_data.156c6e127736.0000000000000033:head by client.5529.0:122 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 62'76)", 2024-04-25T20:10:54.516 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:52.665268+0000", 2024-04-25T20:10:54.516 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 1.8378525830000001, 2024-04-25T20:10:54.516 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0030889789999999999, 2024-04-25T20:10:54.516 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.516 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "reached pg", 2024-04-25T20:10:54.516 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.516 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.516 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.516 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:52.665268+0000", 2024-04-25T20:10:54.516 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.516 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.516 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.516 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.516 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:52.665268+0000", 2024-04-25T20:10:54.516 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.516 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.517 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.517 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.517 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:52.665267+0000", 2024-04-25T20:10:54.517 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:54.517 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.517 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.517 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.517 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:52.665276+0000", 2024-04-25T20:10:54.517 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.8540000000000003e-06 2024-04-25T20:10:54.517 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.517 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.517 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.517 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:52.665288+0000", 2024-04-25T20:10:54.517 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.1887e-05 2024-04-25T20:10:54.518 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.518 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.518 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.518 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:52.665295+0000", 2024-04-25T20:10:54.518 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.7279999999999998e-06 2024-04-25T20:10:54.518 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.518 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.518 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.518 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:52.665340+0000", 2024-04-25T20:10:54.518 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4.5445999999999998e-05 2024-04-25T20:10:54.518 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.518 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.518 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.518 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:52.668357+0000", 2024-04-25T20:10:54.519 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0030166279999999999 2024-04-25T20:10:54.519 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.519 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.519 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.519 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.519 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.519 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "pg_update_log_missing(1.1 epoch 63/21 rep_tid 137 entries 63'49 (0'0) error 1:9da1db08:::rbd_data.141861bceda4.000000000000000c:head by client.5547.0:65 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 62'46)", 2024-04-25T20:10:54.519 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.115511+0000", 2024-04-25T20:10:54.519 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 1.3876097519999999, 2024-04-25T20:10:54.519 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0041036680000000004, 2024-04-25T20:10:54.519 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.519 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "reached pg", 2024-04-25T20:10:54.519 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.519 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.519 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.520 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.115511+0000", 2024-04-25T20:10:54.520 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.520 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.520 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.520 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.520 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.115511+0000", 2024-04-25T20:10:54.520 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.520 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.520 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.520 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.520 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.115510+0000", 2024-04-25T20:10:54.520 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:54.520 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.520 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.520 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.520 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.115516+0000", 2024-04-25T20:10:54.521 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.2090000000000002e-06 2024-04-25T20:10:54.521 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.521 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.521 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.521 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.115529+0000", 2024-04-25T20:10:54.521 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.2303999999999999e-05 2024-04-25T20:10:54.521 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.521 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.521 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.521 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.115535+0000", 2024-04-25T20:10:54.521 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.3990000000000001e-06 2024-04-25T20:10:54.521 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.521 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.521 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.521 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.115571+0000", 2024-04-25T20:10:54.521 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 3.6087999999999997e-05 2024-04-25T20:10:54.522 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.522 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.522 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.522 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.119614+0000", 2024-04-25T20:10:54.522 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0040431690000000001 2024-04-25T20:10:54.522 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.522 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.522 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.522 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.522 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.522 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "pg_update_log_missing(1.4 epoch 63/25 rep_tid 309 entries 63'81 (0'0) error 1:3c6329ca:::rbd_data.141861bceda4.0000000000000019:head by client.5547.0:78 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 63'79)", 2024-04-25T20:10:54.522 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.120520+0000", 2024-04-25T20:10:54.522 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 1.382600692, 2024-04-25T20:10:54.522 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0041441209999999997, 2024-04-25T20:10:54.523 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.523 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "reached pg", 2024-04-25T20:10:54.523 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.523 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.523 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.523 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.120520+0000", 2024-04-25T20:10:54.523 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.523 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.523 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.523 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.523 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.120520+0000", 2024-04-25T20:10:54.523 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.523 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.523 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.523 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.523 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.120519+0000", 2024-04-25T20:10:54.524 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:54.524 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.524 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.524 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.524 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.120528+0000", 2024-04-25T20:10:54.524 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.8769999999999999e-06 2024-04-25T20:10:54.524 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.524 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.524 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.524 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.120544+0000", 2024-04-25T20:10:54.524 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.5522000000000001e-05 2024-04-25T20:10:54.524 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.524 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.524 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.524 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.120554+0000", 2024-04-25T20:10:54.524 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0924000000000001e-05 2024-04-25T20:10:54.525 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.525 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.525 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.525 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.120627+0000", 2024-04-25T20:10:54.525 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.2558999999999994e-05 2024-04-25T20:10:54.525 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.525 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.525 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.525 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.124664+0000", 2024-04-25T20:10:54.525 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0040368590000000003 2024-04-25T20:10:54.525 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.525 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.525 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.525 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.526 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.526 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "pg_update_log_missing(1.1 epoch 63/21 rep_tid 143 entries 63'51 (0'0) error 1:886e0fbc:::rbd_data.141861bceda4.000000000000001a:head by client.5547.0:79 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 63'48)", 2024-04-25T20:10:54.526 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.120746+0000", 2024-04-25T20:10:54.526 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 1.38237415, 2024-04-25T20:10:54.526 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.003917579, 2024-04-25T20:10:54.526 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.526 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "reached pg", 2024-04-25T20:10:54.526 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.526 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.526 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.526 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.120746+0000", 2024-04-25T20:10:54.526 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.526 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.526 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.526 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.527 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.120746+0000", 2024-04-25T20:10:54.527 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.527 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.527 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.527 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.527 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.120746+0000", 2024-04-25T20:10:54.527 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:54.527 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.527 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.527 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.527 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.120754+0000", 2024-04-25T20:10:54.527 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.6920000000000008e-06 2024-04-25T20:10:54.527 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.527 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.527 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.528 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.120767+0000", 2024-04-25T20:10:54.528 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.2651e-05 2024-04-25T20:10:54.528 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.528 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.528 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.528 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.120774+0000", 2024-04-25T20:10:54.528 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.3950000000000003e-06 2024-04-25T20:10:54.528 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.528 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.528 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.528 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.120826+0000", 2024-04-25T20:10:54.528 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.1913999999999998e-05 2024-04-25T20:10:54.528 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.528 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.528 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.528 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.124664+0000", 2024-04-25T20:10:54.529 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0038375549999999999 2024-04-25T20:10:54.529 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.529 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.529 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.529 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.529 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.529 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "pg_update_log_missing(1.4 epoch 63/25 rep_tid 313 entries 63'83 (0'0) error 1:2679d1bd:::rbd_data.141861bceda4.0000000000000032:head by client.5547.0:104 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 63'81)", 2024-04-25T20:10:54.529 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.129207+0000", 2024-04-25T20:10:54.529 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 1.373912993, 2024-04-25T20:10:54.529 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0032010469999999998, 2024-04-25T20:10:54.529 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.529 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "reached pg", 2024-04-25T20:10:54.529 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.529 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.529 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.530 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.129207+0000", 2024-04-25T20:10:54.530 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.530 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.530 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.530 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.530 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.129207+0000", 2024-04-25T20:10:54.530 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.530 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.530 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.530 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.530 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.129207+0000", 2024-04-25T20:10:54.530 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:54.530 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.530 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.530 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.531 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.129215+0000", 2024-04-25T20:10:54.531 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.3520000000000007e-06 2024-04-25T20:10:54.531 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.531 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.531 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.531 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.129233+0000", 2024-04-25T20:10:54.531 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.7498000000000001e-05 2024-04-25T20:10:54.531 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.531 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.531 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.531 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.129243+0000", 2024-04-25T20:10:54.531 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0592e-05 2024-04-25T20:10:54.531 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.531 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.531 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.531 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.129304+0000", 2024-04-25T20:10:54.531 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.0560999999999998e-05 2024-04-25T20:10:54.532 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.532 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.532 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.532 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.132409+0000", 2024-04-25T20:10:54.532 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0031046989999999998 2024-04-25T20:10:54.532 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.532 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.532 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.532 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.532 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.532 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "pg_update_log_missing(1.4 epoch 64/25 rep_tid 344 entries 64'92 (0'0) error 1:3e1353a3:::rbd_data.15c034b083aa.0000000000000013:head by client.5607.0:91 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 64'90)", 2024-04-25T20:10:54.532 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.826441+0000", 2024-04-25T20:10:54.532 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.67667944499999999, 2024-04-25T20:10:54.532 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.003526295, 2024-04-25T20:10:54.532 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.532 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "reached pg", 2024-04-25T20:10:54.532 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.533 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.533 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.533 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.826441+0000", 2024-04-25T20:10:54.533 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.533 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.533 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.533 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.533 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.826441+0000", 2024-04-25T20:10:54.533 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.533 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.533 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.533 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.533 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.826440+0000", 2024-04-25T20:10:54.533 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.999999 2024-04-25T20:10:54.533 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.533 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.533 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.534 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.826451+0000", 2024-04-25T20:10:54.534 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0587e-05 2024-04-25T20:10:54.534 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.534 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.534 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.534 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.826472+0000", 2024-04-25T20:10:54.534 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 2.1263999999999998e-05 2024-04-25T20:10:54.534 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.534 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.534 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.534 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.826485+0000", 2024-04-25T20:10:54.534 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.3123e-05 2024-04-25T20:10:54.534 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.534 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.534 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.535 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.826555+0000", 2024-04-25T20:10:54.535 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.0251000000000002e-05 2024-04-25T20:10:54.535 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.535 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.535 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.535 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.829967+0000", 2024-04-25T20:10:54.535 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.003411931 2024-04-25T20:10:54.535 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.535 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.535 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.535 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.535 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.535 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "pg_update_log_missing(1.4 epoch 64/25 rep_tid 347 entries 64'93 (0'0) error 1:2a68b601:::rbd_data.15c034b083aa.0000000000000015:head by client.5607.0:93 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 64'90)", 2024-04-25T20:10:54.535 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.826968+0000", 2024-04-25T20:10:54.535 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.67615225999999995, 2024-04-25T20:10:54.535 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0068594019999999997, 2024-04-25T20:10:54.535 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.536 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "reached pg", 2024-04-25T20:10:54.536 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.536 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.536 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.536 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.826968+0000", 2024-04-25T20:10:54.536 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.536 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.536 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.536 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.536 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.826968+0000", 2024-04-25T20:10:54.536 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.536 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.536 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.536 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.536 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.826968+0000", 2024-04-25T20:10:54.536 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:54.537 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.537 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.537 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.537 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.826977+0000", 2024-04-25T20:10:54.537 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 9.3389999999999999e-06 2024-04-25T20:10:54.537 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.537 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.537 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.537 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.826995+0000", 2024-04-25T20:10:54.537 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.7672e-05 2024-04-25T20:10:54.537 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.537 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.537 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.537 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.827008+0000", 2024-04-25T20:10:54.537 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.3089e-05 2024-04-25T20:10:54.538 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.538 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.538 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.538 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.827061+0000", 2024-04-25T20:10:54.538 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.3358e-05 2024-04-25T20:10:54.538 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.538 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.538 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.538 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.833828+0000", 2024-04-25T20:10:54.538 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.006766584 2024-04-25T20:10:54.538 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.538 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.538 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.538 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.539 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.539 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "pg_update_log_missing(1.4 epoch 64/25 rep_tid 348 entries 64'94 (0'0) error 1:370c0091:::rbd_data.15c034b083aa.0000000000000017:head by client.5607.0:95 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 64'90)", 2024-04-25T20:10:54.539 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.827265+0000", 2024-04-25T20:10:54.539 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.67585574299999995, 2024-04-25T20:10:54.539 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0065699699999999996, 2024-04-25T20:10:54.539 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.539 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "reached pg", 2024-04-25T20:10:54.539 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.539 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.539 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.539 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.827265+0000", 2024-04-25T20:10:54.539 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.539 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.539 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.539 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.540 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.827265+0000", 2024-04-25T20:10:54.540 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.540 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.540 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.540 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.540 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.827264+0000", 2024-04-25T20:10:54.540 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:54.540 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.540 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.540 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.540 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.827272+0000", 2024-04-25T20:10:54.540 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.1869999999999997e-06 2024-04-25T20:10:54.540 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.540 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.540 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.541 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.827285+0000", 2024-04-25T20:10:54.541 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.2948e-05 2024-04-25T20:10:54.541 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.541 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.541 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.541 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.827296+0000", 2024-04-25T20:10:54.541 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.0815000000000001e-05 2024-04-25T20:10:54.541 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.541 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.541 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.541 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.827351+0000", 2024-04-25T20:10:54.541 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.4580000000000003e-05 2024-04-25T20:10:54.541 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.541 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.541 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.542 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.833835+0000", 2024-04-25T20:10:54.542 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0064839850000000003 2024-04-25T20:10:54.542 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.542 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.542 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.542 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.542 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.542 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "pg_update_log_missing(1.4 epoch 64/25 rep_tid 349 entries 64'95 (0'0) error 1:26bc92e5:::rbd_data.15c034b083aa.000000000000001a:head by client.5607.0:98 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 64'91)", 2024-04-25T20:10:54.542 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.830526+0000", 2024-04-25T20:10:54.542 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.67259480999999999, 2024-04-25T20:10:54.542 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0071522469999999996, 2024-04-25T20:10:54.542 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.542 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "reached pg", 2024-04-25T20:10:54.542 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.542 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.542 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.543 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.830526+0000", 2024-04-25T20:10:54.543 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.543 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.543 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.543 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.543 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.830526+0000", 2024-04-25T20:10:54.543 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.543 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.543 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.543 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.543 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.830525+0000", 2024-04-25T20:10:54.543 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:54.543 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.543 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.543 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.543 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.830532+0000", 2024-04-25T20:10:54.544 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.9970000000000001e-06 2024-04-25T20:10:54.544 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.544 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.544 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.544 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.830544+0000", 2024-04-25T20:10:54.544 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.2251e-05 2024-04-25T20:10:54.544 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.544 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.544 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.544 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.830551+0000", 2024-04-25T20:10:54.544 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.5649999999999998e-06 2024-04-25T20:10:54.544 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.544 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.544 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.544 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.830612+0000", 2024-04-25T20:10:54.544 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.0834000000000001e-05 2024-04-25T20:10:54.544 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.545 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.545 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.545 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.837678+0000", 2024-04-25T20:10:54.545 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0070661600000000001 2024-04-25T20:10:54.545 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.545 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.545 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.545 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.545 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.545 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "pg_update_log_missing(1.4 epoch 64/25 rep_tid 350 entries 64'96 (0'0) error 1:3a53e042:::rbd_data.15c034b083aa.000000000000001b:head by client.5607.0:99 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 64'91)", 2024-04-25T20:10:54.545 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.830895+0000", 2024-04-25T20:10:54.545 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.67222500799999996, 2024-04-25T20:10:54.545 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.006791844, 2024-04-25T20:10:54.545 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.545 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "reached pg", 2024-04-25T20:10:54.545 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.545 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.546 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.546 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.830895+0000", 2024-04-25T20:10:54.546 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.546 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.546 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.546 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.546 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.830895+0000", 2024-04-25T20:10:54.546 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.546 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.546 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.546 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.546 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.830895+0000", 2024-04-25T20:10:54.546 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:54.546 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.546 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.547 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.547 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.830901+0000", 2024-04-25T20:10:54.547 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.5300000000000002e-06 2024-04-25T20:10:54.547 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.547 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.547 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.547 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.830913+0000", 2024-04-25T20:10:54.547 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.169e-05 2024-04-25T20:10:54.547 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.547 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.547 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.547 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.830920+0000", 2024-04-25T20:10:54.547 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.4640000000000003e-06 2024-04-25T20:10:54.547 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.547 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.548 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.548 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.830972+0000", 2024-04-25T20:10:54.548 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.2494000000000003e-05 2024-04-25T20:10:54.548 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.548 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.548 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.548 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.837687+0000", 2024-04-25T20:10:54.548 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0067152269999999998 2024-04-25T20:10:54.548 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.548 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.548 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.548 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.548 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.548 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "pg_update_log_missing(1.1 epoch 64/21 rep_tid 162 entries 64'59 (0'0) error 1:850975e8:::rbd_data.15c034b083aa.000000000000001d:head by client.5607.0:101 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 63'57)", 2024-04-25T20:10:54.549 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.834860+0000", 2024-04-25T20:10:54.549 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.66826080799999998, 2024-04-25T20:10:54.549 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0065846400000000001, 2024-04-25T20:10:54.549 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.549 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "reached pg", 2024-04-25T20:10:54.549 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.549 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.549 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.549 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.834860+0000", 2024-04-25T20:10:54.549 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.549 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.549 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.549 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.549 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.834860+0000", 2024-04-25T20:10:54.550 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.550 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.550 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.550 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.550 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.834859+0000", 2024-04-25T20:10:54.550 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:54.550 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.550 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.550 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.550 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.834865+0000", 2024-04-25T20:10:54.550 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.1720000000000004e-06 2024-04-25T20:10:54.550 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.550 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.550 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.551 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.834878+0000", 2024-04-25T20:10:54.551 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.2811e-05 2024-04-25T20:10:54.551 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.551 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.551 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.551 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.834890+0000", 2024-04-25T20:10:54.551 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.167e-05 2024-04-25T20:10:54.551 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.551 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.551 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.551 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.834974+0000", 2024-04-25T20:10:54.551 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.4503000000000001e-05 2024-04-25T20:10:54.551 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.551 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.551 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.552 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.841444+0000", 2024-04-25T20:10:54.552 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.006469842 2024-04-25T20:10:54.552 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.552 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.552 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.552 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.552 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.552 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "pg_update_log_missing(1.1 epoch 64/21 rep_tid 164 entries 64'60 (0'0) error 1:94139360:::rbd_data.15c034b083aa.000000000000001f:head by client.5607.0:103 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 63'57)", 2024-04-25T20:10:54.552 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.835157+0000", 2024-04-25T20:10:54.552 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.66796366500000004, 2024-04-25T20:10:54.552 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0063062860000000004, 2024-04-25T20:10:54.552 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.552 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "reached pg", 2024-04-25T20:10:54.552 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.553 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.553 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.553 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.835157+0000", 2024-04-25T20:10:54.553 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.553 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.553 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.553 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.553 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.835157+0000", 2024-04-25T20:10:54.553 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.553 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.553 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.553 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.553 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.835156+0000", 2024-04-25T20:10:54.553 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:54.553 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.553 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.554 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.554 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.835164+0000", 2024-04-25T20:10:54.554 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.2660000000000001e-06 2024-04-25T20:10:54.554 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.554 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.554 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.554 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.835180+0000", 2024-04-25T20:10:54.554 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.5988e-05 2024-04-25T20:10:54.554 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.554 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.554 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.554 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.835194+0000", 2024-04-25T20:10:54.554 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.3165e-05 2024-04-25T20:10:54.555 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.555 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.555 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.555 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.835254+0000", 2024-04-25T20:10:54.555 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.0856000000000001e-05 2024-04-25T20:10:54.555 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.555 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.555 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.555 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.841463+0000", 2024-04-25T20:10:54.555 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0062086449999999996 2024-04-25T20:10:54.555 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.555 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.555 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.555 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.556 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.556 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "pg_update_log_missing(1.1 epoch 64/21 rep_tid 165 entries 64'61 (0'0) error 1:9dad303d:::rbd_data.15c034b083aa.0000000000000023:head by client.5607.0:107 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 63'57)", 2024-04-25T20:10:54.556 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.835766+0000", 2024-04-25T20:10:54.556 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.66735483600000001, 2024-04-25T20:10:54.556 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0057040600000000004, 2024-04-25T20:10:54.556 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.556 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "reached pg", 2024-04-25T20:10:54.556 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.556 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.556 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.556 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.835766+0000", 2024-04-25T20:10:54.556 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.556 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.556 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.556 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.556 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.835766+0000", 2024-04-25T20:10:54.557 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.557 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.557 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.557 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.557 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.835765+0000", 2024-04-25T20:10:54.557 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:54.557 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.557 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.557 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.557 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.835771+0000", 2024-04-25T20:10:54.557 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.2129999999999996e-06 2024-04-25T20:10:54.557 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.557 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.557 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.557 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.835783+0000", 2024-04-25T20:10:54.557 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.2087e-05 2024-04-25T20:10:54.557 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.558 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.558 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.558 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.835792+0000", 2024-04-25T20:10:54.558 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.3229999999999998e-06 2024-04-25T20:10:54.558 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.558 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.558 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.558 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.835845+0000", 2024-04-25T20:10:54.558 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.3177999999999998e-05 2024-04-25T20:10:54.558 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.558 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.558 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.558 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.841470+0000", 2024-04-25T20:10:54.558 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0056248080000000002 2024-04-25T20:10:54.558 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.559 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.559 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.559 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.559 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.559 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "pg_update_log_missing(1.4 epoch 64/25 rep_tid 351 entries 64'97 (0'0) error 1:2db7e005:::rbd_data.15c034b083aa.0000000000000025:head by client.5607.0:109 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 64'95)", 2024-04-25T20:10:54.559 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.838471+0000", 2024-04-25T20:10:54.559 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.66464921399999999, 2024-04-25T20:10:54.559 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0033134710000000001, 2024-04-25T20:10:54.559 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.559 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "reached pg", 2024-04-25T20:10:54.559 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.559 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.559 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.559 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.838471+0000", 2024-04-25T20:10:54.559 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.559 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.560 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.560 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.560 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.838471+0000", 2024-04-25T20:10:54.560 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.560 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.560 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.560 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.560 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.838470+0000", 2024-04-25T20:10:54.560 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.999999 2024-04-25T20:10:54.560 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.560 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.560 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.560 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.838479+0000", 2024-04-25T20:10:54.560 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.6270000000000006e-06 2024-04-25T20:10:54.560 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.561 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.561 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.561 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.838498+0000", 2024-04-25T20:10:54.561 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.8391000000000001e-05 2024-04-25T20:10:54.561 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.561 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.561 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.561 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.838509+0000", 2024-04-25T20:10:54.561 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.1939e-05 2024-04-25T20:10:54.561 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.561 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.561 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.561 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.838584+0000", 2024-04-25T20:10:54.561 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 7.4365999999999999e-05 2024-04-25T20:10:54.561 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.561 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.562 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.562 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.841785+0000", 2024-04-25T20:10:54.562 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0032009199999999999 2024-04-25T20:10:54.562 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.562 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.562 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.562 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.562 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.562 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "pg_update_log_missing(1.1 epoch 64/21 rep_tid 166 entries 64'62 (0'0) error 1:88d2078a:::rbd_data.15c034b083aa.000000000000002c:head by client.5607.0:116 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 64'60)", 2024-04-25T20:10:54.562 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.842203+0000", 2024-04-25T20:10:54.562 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.66091729399999999, 2024-04-25T20:10:54.562 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0032668979999999999, 2024-04-25T20:10:54.562 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.562 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "reached pg", 2024-04-25T20:10:54.562 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.563 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.563 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.563 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.842203+0000", 2024-04-25T20:10:54.563 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.563 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.563 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.563 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.563 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.842203+0000", 2024-04-25T20:10:54.563 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.563 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.563 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.563 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.563 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.842203+0000", 2024-04-25T20:10:54.563 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:54.563 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.563 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.563 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.564 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.842215+0000", 2024-04-25T20:10:54.564 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.2473999999999999e-05 2024-04-25T20:10:54.564 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.564 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.564 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.564 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.842234+0000", 2024-04-25T20:10:54.564 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.8479e-05 2024-04-25T20:10:54.564 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.564 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.564 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.564 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.842245+0000", 2024-04-25T20:10:54.564 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.1100999999999999e-05 2024-04-25T20:10:54.564 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.564 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.564 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.564 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.842312+0000", 2024-04-25T20:10:54.564 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 6.7745999999999998e-05 2024-04-25T20:10:54.565 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.565 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.565 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.565 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.845470+0000", 2024-04-25T20:10:54.565 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.003157637 2024-04-25T20:10:54.565 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.565 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.565 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.565 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.565 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.565 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "pg_update_log_missing(1.4 epoch 64/25 rep_tid 353 entries 64'98 (0'0) error 1:223cd586:::rbd_data.15c034b083aa.000000000000002f:head by client.5607.0:119 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 64'96)", 2024-04-25T20:10:54.565 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.842722+0000", 2024-04-25T20:10:54.565 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.66039824300000005, 2024-04-25T20:10:54.565 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0065617139999999997, 2024-04-25T20:10:54.565 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.565 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "reached pg", 2024-04-25T20:10:54.566 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.566 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.566 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.566 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.842722+0000", 2024-04-25T20:10:54.566 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.566 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.566 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.566 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.566 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.842722+0000", 2024-04-25T20:10:54.566 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.566 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.566 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.566 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.566 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.842722+0000", 2024-04-25T20:10:54.566 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:54.566 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.567 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.567 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.567 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.842730+0000", 2024-04-25T20:10:54.567 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 8.4929999999999998e-06 2024-04-25T20:10:54.567 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.567 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.567 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.567 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.842747+0000", 2024-04-25T20:10:54.567 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.6880000000000001e-05 2024-04-25T20:10:54.567 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.567 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.567 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.567 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.842758+0000", 2024-04-25T20:10:54.567 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.1155000000000001e-05 2024-04-25T20:10:54.567 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.567 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.568 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.568 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.842813+0000", 2024-04-25T20:10:54.568 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.4833999999999997e-05 2024-04-25T20:10:54.568 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.568 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.568 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.568 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.849284+0000", 2024-04-25T20:10:54.568 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0064710119999999999 2024-04-25T20:10:54.568 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.568 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.568 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.568 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.568 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.569 INFO:teuthology.orchestra.run.smithi135.stdout: "description": "pg_update_log_missing(1.4 epoch 64/25 rep_tid 357 entries 64'100 (0'0) error 1:2299e625:::rbd_data.15c034b083aa.000000000000003c:head by client.5607.0:132 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 64'97)", 2024-04-25T20:10:54.569 INFO:teuthology.orchestra.run.smithi135.stdout: "initiated_at": "2024-04-25T20:10:53.850475+0000", 2024-04-25T20:10:54.569 INFO:teuthology.orchestra.run.smithi135.stdout: "age": 0.65264503900000004, 2024-04-25T20:10:54.569 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0049790160000000002, 2024-04-25T20:10:54.569 INFO:teuthology.orchestra.run.smithi135.stdout: "type_data": { 2024-04-25T20:10:54.569 INFO:teuthology.orchestra.run.smithi135.stdout: "flag_point": "reached pg", 2024-04-25T20:10:54.569 INFO:teuthology.orchestra.run.smithi135.stdout: "events": [ 2024-04-25T20:10:54.569 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.569 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "initiated", 2024-04-25T20:10:54.569 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.850475+0000", 2024-04-25T20:10:54.569 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.569 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.569 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.569 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "throttled", 2024-04-25T20:10:54.569 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.850475+0000", 2024-04-25T20:10:54.569 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0 2024-04-25T20:10:54.570 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.570 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.570 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "header_read", 2024-04-25T20:10:54.570 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.850475+0000", 2024-04-25T20:10:54.570 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 4294967295.9999995 2024-04-25T20:10:54.570 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.570 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.570 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "all_read", 2024-04-25T20:10:54.570 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.850491+0000", 2024-04-25T20:10:54.570 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.6209999999999999e-05 2024-04-25T20:10:54.570 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.570 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.570 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "dispatched", 2024-04-25T20:10:54.570 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.850506+0000", 2024-04-25T20:10:54.571 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.5258e-05 2024-04-25T20:10:54.571 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.571 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.571 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "queued_for_pg", 2024-04-25T20:10:54.571 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.850518+0000", 2024-04-25T20:10:54.571 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 1.1270000000000001e-05 2024-04-25T20:10:54.571 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.571 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.571 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "reached_pg", 2024-04-25T20:10:54.571 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.850572+0000", 2024-04-25T20:10:54.571 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 5.4023000000000001e-05 2024-04-25T20:10:54.571 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-25T20:10:54.571 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-25T20:10:54.571 INFO:teuthology.orchestra.run.smithi135.stdout: "event": "done", 2024-04-25T20:10:54.571 INFO:teuthology.orchestra.run.smithi135.stdout: "time": "2024-04-25T20:10:53.855454+0000", 2024-04-25T20:10:54.571 INFO:teuthology.orchestra.run.smithi135.stdout: "duration": 0.0048828830000000002 2024-04-25T20:10:54.572 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.572 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.572 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.572 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-25T20:10:54.572 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-25T20:10:54.572 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-25T20:10:54.573 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-25T20:10:54.580 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0: {} 2024-04-25T20:10:54.580 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-25T20:10:54.597 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1: {} 2024-04-25T20:10:54.598 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-25T20:10:54.614 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2: {} 2024-04-25T20:10:54.614 INFO:teuthology.orchestra.run.smithi039.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-25T20:10:54.615 INFO:tasks.ceph.osd.7.smithi135.stderr:2024-04-25T20:10:54.615+0000 7f6f2a971640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31720) UID: 0 2024-04-25T20:10:54.631 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3: {} 2024-04-25T20:10:54.631 INFO:teuthology.orchestra.run.smithi039.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-25T20:10:54.649 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4: {} 2024-04-25T20:10:54.649 INFO:teuthology.orchestra.run.smithi039.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-25T20:10:54.665 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5: {} 2024-04-25T20:10:54.665 INFO:teuthology.orchestra.run.smithi039.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-25T20:10:54.682 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6: {} 2024-04-25T20:10:54.682 INFO:teuthology.orchestra.run.smithi039.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-25T20:10:54.694 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-04-25T20:10:54.693+0000 7f62100e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31807) UID: 0 2024-04-25T20:10:54.695 DEBUG:teuthology.parallel:result is None 2024-04-25T20:10:54.695 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T20:10:54.700 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7: {} 2024-04-25T20:10:54.701 INFO:teuthology.orchestra.run.smithi039.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-25T20:10:54.718 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:54.718 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:54.724 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T20:10:54.724 DEBUG:teuthology.orchestra.run.smithi039:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-04-25T20:10:54.727 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-25T20:10:54.782 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T20:10:54.782 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-04-25T20:10:54.791 INFO:tasks.thrashosds:joining thrashosds 2024-04-25T20:10:54.827 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:54.827 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:54.836 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-25T20:10:54.937 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:54.937 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:54.946 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-25T20:10:55.048 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:55.048 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:55.057 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-25T20:10:55.158 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:55.158 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:55.167 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-25T20:10:55.177 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is set 2024-04-25T20:10:55.269 INFO:teuthology.orchestra.run.smithi135.stdout:ERROR: (22) Invalid argument 2024-04-25T20:10:55.269 INFO:teuthology.orchestra.run.smithi135.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-25T20:10:57.192 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-25T20:10:58.051 INFO:tasks.thrashosds.thrasher:inning osd 2024-04-25T20:10:58.051 INFO:tasks.thrashosds.thrasher:Adding osd 3 2024-04-25T20:10:58.051 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 3 2024-04-25T20:10:58.171 INFO:teuthology.orchestra.run.smithi039.stderr:noscrub is unset 2024-04-25T20:10:58.184 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-25T20:10:58.365 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-04-25T20:10:58.364+0000 7f249d8be640 -1 mon.a@0(leader).osd e66 definitely_dead 0 2024-04-25T20:10:59.222 INFO:teuthology.orchestra.run.smithi039.stderr:marked in osd.3. 2024-04-25T20:10:59.224 INFO:teuthology.orchestra.run.smithi039.stderr:nodeep-scrub is unset 2024-04-25T20:10:59.241 INFO:tasks.thrashosds.thrasher:Added osd 3 2024-04-25T20:10:59.241 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-04-25T20:10:59.403 INFO:teuthology.orchestra.run.smithi039.stdout:mon.a: {} 2024-04-25T20:10:59.404 INFO:teuthology.orchestra.run.smithi039.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-25T20:10:59.414 INFO:teuthology.orchestra.run.smithi039.stdout:mon.b: {} 2024-04-25T20:10:59.414 INFO:teuthology.orchestra.run.smithi039.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-25T20:10:59.423 INFO:teuthology.orchestra.run.smithi039.stdout:mon.c: {} 2024-04-25T20:10:59.423 INFO:teuthology.orchestra.run.smithi039.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-25T20:10:59.434 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-25T20:11:00.232 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.0 to 1 (10000) 2024-04-25T20:11:00.246 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-04-25T20:11:01.247 INFO:teuthology.orchestra.run.smithi039.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-04-25T20:11:01.262 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-25T20:11:02.259 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.1 to 1 (10000) 2024-04-25T20:11:02.273 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-04-25T20:11:03.255 INFO:teuthology.orchestra.run.smithi039.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-04-25T20:11:03.274 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-25T20:11:04.284 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.2 to 1 (10000) 2024-04-25T20:11:04.298 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-04-25T20:11:05.285 INFO:teuthology.orchestra.run.smithi039.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-04-25T20:11:05.301 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-25T20:11:06.305 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.3 to 1 (10000) 2024-04-25T20:11:06.322 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-04-25T20:11:07.332 INFO:teuthology.orchestra.run.smithi039.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-04-25T20:11:07.350 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-04-25T20:11:08.335 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.4 to 1 (10000) 2024-04-25T20:11:08.351 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-04-25T20:11:09.352 INFO:teuthology.orchestra.run.smithi039.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-04-25T20:11:09.369 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-04-25T20:11:10.365 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.5 to 1 (10000) 2024-04-25T20:11:10.381 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-04-25T20:11:11.373 INFO:teuthology.orchestra.run.smithi039.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-04-25T20:11:11.390 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-04-25T20:11:12.393 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.6 to 1 (10000) 2024-04-25T20:11:12.409 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-04-25T20:11:13.393 INFO:teuthology.orchestra.run.smithi039.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-04-25T20:11:13.410 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-04-25T20:11:14.401 INFO:teuthology.orchestra.run.smithi039.stderr:reweighted osd.7 to 1 (10000) 2024-04-25T20:11:14.418 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-04-25T20:11:15.422 INFO:teuthology.orchestra.run.smithi039.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-04-25T20:11:15.436 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-04-25T20:11:15.437 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-04-25T20:11:15.437 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-04-25T20:11:15.437 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-04-25T20:11:15.437 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T20:11:15.437 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T20:11:15.730 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:11:15.731 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":83,"fsid":"975ceafd-555d-4f71-81e4-1f70364da00f","created":"2024-04-25T20:09:03.377320+0000","modified":"2024-04-25T20:11:15.405214+0000","last_up_change":"2024-04-25T20:09:13.740341+0000","last_in_change":"2024-04-25T20:10:59.204972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-25T20:09:14.202057+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":7,"snap_epoch":63,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-04-25T20:09:15.645688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7a7118ba-4aee-404f-97e3-94c50e2996c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":31832},{"type":"v1","addr":"172.21.15.39:6818","nonce":31832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":31832},{"type":"v1","addr":"172.21.15.39:6822","nonce":31832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":31832},{"type":"v1","addr":"172.21.15.39:6830","nonce":31832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":31832},{"type":"v1","addr":"172.21.15.39:6826","nonce":31832}]},"public_addr":"172.21.15.39:6818/31832","cluster_addr":"172.21.15.39:6822/31832","heartbeat_back_addr":"172.21.15.39:6830/31832","heartbeat_front_addr":"172.21.15.39:6826/31832","state":["exists","up"]},{"osd":1,"uuid":"13637dbe-e37f-457f-b68b-8eed1b970617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":31831},{"type":"v1","addr":"172.21.15.39:6809","nonce":31831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":31831},{"type":"v1","addr":"172.21.15.39:6811","nonce":31831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":31831},{"type":"v1","addr":"172.21.15.39:6815","nonce":31831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":31831},{"type":"v1","addr":"172.21.15.39:6813","nonce":31831}]},"public_addr":"172.21.15.39:6809/31831","cluster_addr":"172.21.15.39:6811/31831","heartbeat_back_addr":"172.21.15.39:6815/31831","heartbeat_front_addr":"172.21.15.39:6813/31831","state":["exists","up"]},{"osd":2,"uuid":"29e20ce2-4776-455d-8cf4-d539e738a6a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":31829},{"type":"v1","addr":"172.21.15.39:6801","nonce":31829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":31829},{"type":"v1","addr":"172.21.15.39:6803","nonce":31829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":31829},{"type":"v1","addr":"172.21.15.39:6807","nonce":31829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":31829},{"type":"v1","addr":"172.21.15.39:6805","nonce":31829}]},"public_addr":"172.21.15.39:6801/31829","cluster_addr":"172.21.15.39:6803/31829","heartbeat_back_addr":"172.21.15.39:6807/31829","heartbeat_front_addr":"172.21.15.39:6805/31829","state":["exists","up"]},{"osd":3,"uuid":"a8b34e02-0898-4303-9fd5-62cbba9fec5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6817","nonce":31830},{"type":"v1","addr":"172.21.15.39:6819","nonce":31830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6821","nonce":31830},{"type":"v1","addr":"172.21.15.39:6823","nonce":31830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6829","nonce":31830},{"type":"v1","addr":"172.21.15.39:6831","nonce":31830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6825","nonce":31830},{"type":"v1","addr":"172.21.15.39:6827","nonce":31830}]},"public_addr":"172.21.15.39:6819/31830","cluster_addr":"172.21.15.39:6823/31830","heartbeat_back_addr":"172.21.15.39:6831/31830","heartbeat_front_addr":"172.21.15.39:6827/31830","state":["exists","up"]},{"osd":4,"uuid":"e92b5c67-6908-4fd4-9940-5cb98b5ee847","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":31724},{"type":"v1","addr":"172.21.15.135:6801","nonce":31724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":31724},{"type":"v1","addr":"172.21.15.135:6803","nonce":31724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":31724},{"type":"v1","addr":"172.21.15.135:6807","nonce":31724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":31724},{"type":"v1","addr":"172.21.15.135:6805","nonce":31724}]},"public_addr":"172.21.15.135:6801/31724","cluster_addr":"172.21.15.135:6803/31724","heartbeat_back_addr":"172.21.15.135:6807/31724","heartbeat_front_addr":"172.21.15.135:6805/31724","state":["exists","up"]},{"osd":5,"uuid":"d74d8fef-0211-413b-bff2-f97776fc1faa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":31734},{"type":"v1","addr":"172.21.15.135:6822","nonce":31734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":31734},{"type":"v1","addr":"172.21.15.135:6826","nonce":31734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":31734},{"type":"v1","addr":"172.21.15.135:6831","nonce":31734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":31734},{"type":"v1","addr":"172.21.15.135:6829","nonce":31734}]},"public_addr":"172.21.15.135:6822/31734","cluster_addr":"172.21.15.135:6826/31734","heartbeat_back_addr":"172.21.15.135:6831/31734","heartbeat_front_addr":"172.21.15.135:6829/31734","state":["exists","up"]},{"osd":6,"uuid":"7dc84c6e-76d8-4f22-8b17-51f62765762d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":31725},{"type":"v1","addr":"172.21.15.135:6809","nonce":31725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":31725},{"type":"v1","addr":"172.21.15.135:6811","nonce":31725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":31725},{"type":"v1","addr":"172.21.15.135:6815","nonce":31725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":31725},{"type":"v1","addr":"172.21.15.135:6813","nonce":31725}]},"public_addr":"172.21.15.135:6809/31725","cluster_addr":"172.21.15.135:6811/31725","heartbeat_back_addr":"172.21.15.135:6815/31725","heartbeat_front_addr":"172.21.15.135:6813/31725","state":["exists","up"]},{"osd":7,"uuid":"7adb3187-aed3-48af-868c-7230e9bfc85e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":31737},{"type":"v1","addr":"172.21.15.135:6817","nonce":31737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":31737},{"type":"v1","addr":"172.21.15.135:6819","nonce":31737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":31737},{"type":"v1","addr":"172.21.15.135:6827","nonce":31737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":31737},{"type":"v1","addr":"172.21.15.135:6823","nonce":31737}]},"public_addr":"172.21.15.135:6817/31737","cluster_addr":"172.21.15.135:6819/31737","heartbeat_back_addr":"172.21.15.135:6827/31737","heartbeat_front_addr":"172.21.15.135:6823/31737","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.683465+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.722423+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.712691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.673892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.768469+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.783897+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.686833+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.739675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"1.2","osds":[0]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T20:11:15.743 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T20:11:15.744 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T20:11:16.039 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:11:16.040 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":83,"fsid":"975ceafd-555d-4f71-81e4-1f70364da00f","created":"2024-04-25T20:09:03.377320+0000","modified":"2024-04-25T20:11:15.405214+0000","last_up_change":"2024-04-25T20:09:13.740341+0000","last_in_change":"2024-04-25T20:10:59.204972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-25T20:09:14.202057+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":7,"snap_epoch":63,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-04-25T20:09:15.645688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7a7118ba-4aee-404f-97e3-94c50e2996c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":31832},{"type":"v1","addr":"172.21.15.39:6818","nonce":31832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":31832},{"type":"v1","addr":"172.21.15.39:6822","nonce":31832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":31832},{"type":"v1","addr":"172.21.15.39:6830","nonce":31832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":31832},{"type":"v1","addr":"172.21.15.39:6826","nonce":31832}]},"public_addr":"172.21.15.39:6818/31832","cluster_addr":"172.21.15.39:6822/31832","heartbeat_back_addr":"172.21.15.39:6830/31832","heartbeat_front_addr":"172.21.15.39:6826/31832","state":["exists","up"]},{"osd":1,"uuid":"13637dbe-e37f-457f-b68b-8eed1b970617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":31831},{"type":"v1","addr":"172.21.15.39:6809","nonce":31831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":31831},{"type":"v1","addr":"172.21.15.39:6811","nonce":31831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":31831},{"type":"v1","addr":"172.21.15.39:6815","nonce":31831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":31831},{"type":"v1","addr":"172.21.15.39:6813","nonce":31831}]},"public_addr":"172.21.15.39:6809/31831","cluster_addr":"172.21.15.39:6811/31831","heartbeat_back_addr":"172.21.15.39:6815/31831","heartbeat_front_addr":"172.21.15.39:6813/31831","state":["exists","up"]},{"osd":2,"uuid":"29e20ce2-4776-455d-8cf4-d539e738a6a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":31829},{"type":"v1","addr":"172.21.15.39:6801","nonce":31829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":31829},{"type":"v1","addr":"172.21.15.39:6803","nonce":31829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":31829},{"type":"v1","addr":"172.21.15.39:6807","nonce":31829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":31829},{"type":"v1","addr":"172.21.15.39:6805","nonce":31829}]},"public_addr":"172.21.15.39:6801/31829","cluster_addr":"172.21.15.39:6803/31829","heartbeat_back_addr":"172.21.15.39:6807/31829","heartbeat_front_addr":"172.21.15.39:6805/31829","state":["exists","up"]},{"osd":3,"uuid":"a8b34e02-0898-4303-9fd5-62cbba9fec5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6817","nonce":31830},{"type":"v1","addr":"172.21.15.39:6819","nonce":31830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6821","nonce":31830},{"type":"v1","addr":"172.21.15.39:6823","nonce":31830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6829","nonce":31830},{"type":"v1","addr":"172.21.15.39:6831","nonce":31830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6825","nonce":31830},{"type":"v1","addr":"172.21.15.39:6827","nonce":31830}]},"public_addr":"172.21.15.39:6819/31830","cluster_addr":"172.21.15.39:6823/31830","heartbeat_back_addr":"172.21.15.39:6831/31830","heartbeat_front_addr":"172.21.15.39:6827/31830","state":["exists","up"]},{"osd":4,"uuid":"e92b5c67-6908-4fd4-9940-5cb98b5ee847","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":31724},{"type":"v1","addr":"172.21.15.135:6801","nonce":31724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":31724},{"type":"v1","addr":"172.21.15.135:6803","nonce":31724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":31724},{"type":"v1","addr":"172.21.15.135:6807","nonce":31724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":31724},{"type":"v1","addr":"172.21.15.135:6805","nonce":31724}]},"public_addr":"172.21.15.135:6801/31724","cluster_addr":"172.21.15.135:6803/31724","heartbeat_back_addr":"172.21.15.135:6807/31724","heartbeat_front_addr":"172.21.15.135:6805/31724","state":["exists","up"]},{"osd":5,"uuid":"d74d8fef-0211-413b-bff2-f97776fc1faa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":31734},{"type":"v1","addr":"172.21.15.135:6822","nonce":31734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":31734},{"type":"v1","addr":"172.21.15.135:6826","nonce":31734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":31734},{"type":"v1","addr":"172.21.15.135:6831","nonce":31734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":31734},{"type":"v1","addr":"172.21.15.135:6829","nonce":31734}]},"public_addr":"172.21.15.135:6822/31734","cluster_addr":"172.21.15.135:6826/31734","heartbeat_back_addr":"172.21.15.135:6831/31734","heartbeat_front_addr":"172.21.15.135:6829/31734","state":["exists","up"]},{"osd":6,"uuid":"7dc84c6e-76d8-4f22-8b17-51f62765762d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":31725},{"type":"v1","addr":"172.21.15.135:6809","nonce":31725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":31725},{"type":"v1","addr":"172.21.15.135:6811","nonce":31725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":31725},{"type":"v1","addr":"172.21.15.135:6815","nonce":31725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":31725},{"type":"v1","addr":"172.21.15.135:6813","nonce":31725}]},"public_addr":"172.21.15.135:6809/31725","cluster_addr":"172.21.15.135:6811/31725","heartbeat_back_addr":"172.21.15.135:6815/31725","heartbeat_front_addr":"172.21.15.135:6813/31725","state":["exists","up"]},{"osd":7,"uuid":"7adb3187-aed3-48af-868c-7230e9bfc85e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":31737},{"type":"v1","addr":"172.21.15.135:6817","nonce":31737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":31737},{"type":"v1","addr":"172.21.15.135:6819","nonce":31737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":31737},{"type":"v1","addr":"172.21.15.135:6827","nonce":31737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":31737},{"type":"v1","addr":"172.21.15.135:6823","nonce":31737}]},"public_addr":"172.21.15.135:6817/31737","cluster_addr":"172.21.15.135:6819/31737","heartbeat_back_addr":"172.21.15.135:6827/31737","heartbeat_front_addr":"172.21.15.135:6823/31737","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.683465+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.722423+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.712691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.673892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.768469+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.783897+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.686833+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.739675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"1.2","osds":[0]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T20:11:16.053 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T20:11:16.054 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T20:11:16.054 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T20:11:16.055 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T20:11:16.055 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T20:11:16.055 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T20:11:16.056 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T20:11:16.056 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T20:11:16.304 INFO:teuthology.orchestra.run.smithi039.stdout:55834574877 2024-04-25T20:11:16.304 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T20:11:16.305 INFO:teuthology.orchestra.run.smithi039.stdout:55834574876 2024-04-25T20:11:16.305 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T20:11:16.308 INFO:teuthology.orchestra.run.smithi039.stdout:55834574876 2024-04-25T20:11:16.308 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T20:11:16.309 INFO:teuthology.orchestra.run.smithi039.stdout:55834574876 2024-04-25T20:11:16.310 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T20:11:16.311 INFO:teuthology.orchestra.run.smithi039.stdout:55834574876 2024-04-25T20:11:16.311 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T20:11:16.312 INFO:teuthology.orchestra.run.smithi039.stdout:55834574876 2024-04-25T20:11:16.312 INFO:teuthology.orchestra.run.smithi039.stdout:55834574876 2024-04-25T20:11:16.313 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T20:11:16.313 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T20:11:16.314 INFO:teuthology.orchestra.run.smithi039.stdout:55834574876 2024-04-25T20:11:16.315 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T20:11:16.894 INFO:teuthology.orchestra.run.smithi039.stdout:55834574875 2024-04-25T20:11:16.907 INFO:teuthology.orchestra.run.smithi039.stdout:55834574876 2024-04-25T20:11:16.914 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574876 got 55834574875 for osd.1 2024-04-25T20:11:16.930 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574877 got 55834574876 for osd.3 2024-04-25T20:11:16.947 INFO:teuthology.orchestra.run.smithi039.stdout:55834574875 2024-04-25T20:11:16.947 INFO:teuthology.orchestra.run.smithi039.stdout:55834574875 2024-04-25T20:11:16.962 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574876 got 55834574875 for osd.0 2024-04-25T20:11:16.963 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574876 got 55834574875 for osd.2 2024-04-25T20:11:16.967 INFO:teuthology.orchestra.run.smithi039.stdout:55834574875 2024-04-25T20:11:16.972 INFO:teuthology.orchestra.run.smithi039.stdout:55834574875 2024-04-25T20:11:16.987 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574876 got 55834574875 for osd.5 2024-04-25T20:11:16.988 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574876 got 55834574875 for osd.4 2024-04-25T20:11:16.988 INFO:teuthology.orchestra.run.smithi039.stdout:55834574875 2024-04-25T20:11:16.988 INFO:teuthology.orchestra.run.smithi039.stdout:55834574875 2024-04-25T20:11:17.002 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574876 got 55834574875 for osd.7 2024-04-25T20:11:17.002 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574876 got 55834574875 for osd.6 2024-04-25T20:11:17.915 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T20:11:17.930 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T20:11:17.963 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T20:11:17.963 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T20:11:17.988 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T20:11:17.988 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T20:11:18.002 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T20:11:18.003 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T20:11:18.470 INFO:teuthology.orchestra.run.smithi039.stdout:55834574876 2024-04-25T20:11:18.488 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574876 got 55834574876 for osd.1 2024-04-25T20:11:18.488 DEBUG:teuthology.parallel:result is None 2024-04-25T20:11:18.542 INFO:teuthology.orchestra.run.smithi039.stdout:55834574876 2024-04-25T20:11:18.556 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574876 got 55834574876 for osd.5 2024-04-25T20:11:18.556 DEBUG:teuthology.parallel:result is None 2024-04-25T20:11:18.560 INFO:teuthology.orchestra.run.smithi039.stdout:55834574877 2024-04-25T20:11:18.575 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574877 got 55834574877 for osd.3 2024-04-25T20:11:18.575 DEBUG:teuthology.parallel:result is None 2024-04-25T20:11:18.579 INFO:teuthology.orchestra.run.smithi039.stdout:55834574876 2024-04-25T20:11:18.593 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574876 got 55834574876 for osd.2 2024-04-25T20:11:18.593 DEBUG:teuthology.parallel:result is None 2024-04-25T20:11:18.615 INFO:teuthology.orchestra.run.smithi039.stdout:55834574876 2024-04-25T20:11:18.624 INFO:teuthology.orchestra.run.smithi039.stdout:55834574876 2024-04-25T20:11:18.629 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574876 got 55834574876 for osd.0 2024-04-25T20:11:18.629 DEBUG:teuthology.parallel:result is None 2024-04-25T20:11:18.639 INFO:teuthology.orchestra.run.smithi039.stdout:55834574876 2024-04-25T20:11:18.642 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574876 got 55834574876 for osd.4 2024-04-25T20:11:18.642 DEBUG:teuthology.parallel:result is None 2024-04-25T20:11:18.644 INFO:teuthology.orchestra.run.smithi039.stdout:55834574876 2024-04-25T20:11:18.653 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574876 got 55834574876 for osd.7 2024-04-25T20:11:18.654 DEBUG:teuthology.parallel:result is None 2024-04-25T20:11:18.657 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574876 got 55834574876 for osd.6 2024-04-25T20:11:18.657 DEBUG:teuthology.parallel:result is None 2024-04-25T20:11:18.657 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-25T20:11:18.658 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T20:11:18.987 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:11:18.987 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-04-25T20:11:18.999 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-04-25T20:11:18.459666+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2418,"num_read_kb":2307,"num_write":538,"num_write_kb":2800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":979,"ondisk_log_size":979,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248512,"kb_used_data":7740,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754726208,"statfs":{"total":773094113280,"available":772839636992,"internally_reserved":0,"allocated":7925760,"data_stored":4497798,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12741,"internal_metadata":246402619},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":-539,"num_read_kb":-490,"num_write":-41,"num_write_kb":-44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.060181"},"pg_stats":[{"pgid":"1.7","version":"64'117","reported_seq":322,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T20:11:15.198251+0000","last_change":"2024-04-25T20:11:03.266483+0000","last_active":"2024-04-25T20:11:15.198251+0000","last_peered":"2024-04-25T20:11:15.198251+0000","last_clean":"2024-04-25T20:11:15.198251+0000","last_became_active":"2024-04-25T20:11:03.266120+0000","last_became_peered":"2024-04-25T20:11:03.266120+0000","last_unstale":"2024-04-25T20:11:15.198251+0000","last_undegraded":"2024-04-25T20:11:15.198251+0000","last_fullsized":"2024-04-25T20:11:15.198251+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:36.703817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033361900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":159,"num_read_kb":209,"num_write":53,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"64'156","reported_seq":500,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T20:11:15.198216+0000","last_change":"2024-04-25T20:10:53.077262+0000","last_active":"2024-04-25T20:11:15.198216+0000","last_peered":"2024-04-25T20:11:15.198216+0000","last_clean":"2024-04-25T20:11:15.198216+0000","last_became_active":"2024-04-25T20:09:16.140616+0000","last_became_peered":"2024-04-25T20:09:16.140616+0000","last_unstale":"2024-04-25T20:11:15.198216+0000","last_undegraded":"2024-04-25T20:11:15.198216+0000","last_fullsized":"2024-04-25T20:11:15.198216+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:31.562175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000351966,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":530,"num_read_kb":465,"num_write":90,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"64'104","reported_seq":419,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T20:11:14.403343+0000","last_change":"2024-04-25T20:11:08.336815+0000","last_active":"2024-04-25T20:11:14.403343+0000","last_peered":"2024-04-25T20:11:14.403343+0000","last_clean":"2024-04-25T20:11:14.403343+0000","last_became_active":"2024-04-25T20:11:08.336485+0000","last_became_peered":"2024-04-25T20:11:08.336485+0000","last_unstale":"2024-04-25T20:11:14.403343+0000","last_undegraded":"2024-04-25T20:11:14.403343+0000","last_fullsized":"2024-04-25T20:11:14.403343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:37.372937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023598099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":408,"num_read_kb":340,"num_write":32,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"64'113","reported_seq":236,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T20:11:14.404154+0000","last_change":"2024-04-25T20:11:03.270954+0000","last_active":"2024-04-25T20:11:14.404154+0000","last_peered":"2024-04-25T20:11:14.404154+0000","last_clean":"2024-04-25T20:11:14.404154+0000","last_became_active":"2024-04-25T20:11:03.270477+0000","last_became_peered":"2024-04-25T20:11:03.270477+0000","last_unstale":"2024-04-25T20:11:14.404154+0000","last_undegraded":"2024-04-25T20:11:14.404154+0000","last_fullsized":"2024-04-25T20:11:14.404154+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:53.502563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024956199999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":99,"num_read_kb":219,"num_write":57,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":222,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T20:11:14.407982+0000","last_change":"2024-04-25T20:09:17.121148+0000","last_active":"2024-04-25T20:11:14.407982+0000","last_peered":"2024-04-25T20:11:14.407982+0000","last_clean":"2024-04-25T20:11:14.407982+0000","last_became_active":"2024-04-25T20:09:17.120312+0000","last_became_peered":"2024-04-25T20:09:17.120312+0000","last_unstale":"2024-04-25T20:11:14.407982+0000","last_undegraded":"2024-04-25T20:11:14.407982+0000","last_fullsized":"2024-04-25T20:11:14.407982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_clean_scrub_stamp":"2024-04-25T20:09:15.745353+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:48.383841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"64'119","reported_seq":323,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T20:11:15.198041+0000","last_change":"2024-04-25T20:10:53.077437+0000","last_active":"2024-04-25T20:11:15.198041+0000","last_peered":"2024-04-25T20:11:15.198041+0000","last_clean":"2024-04-25T20:11:15.198041+0000","last_became_active":"2024-04-25T20:09:15.772551+0000","last_became_peered":"2024-04-25T20:09:15.772551+0000","last_unstale":"2024-04-25T20:11:15.198041+0000","last_undegraded":"2024-04-25T20:11:15.198041+0000","last_fullsized":"2024-04-25T20:11:15.198041+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:55.101025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035253999999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":240,"num_read_kb":216,"num_write":49,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"unknown","last_fresh":"2024-04-25T20:11:14.407530+0000","last_change":"2024-04-25T20:11:14.647838+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T20:11:14.407530+0000","last_undegraded":"2024-04-25T20:11:14.407530+0000","last_fullsized":"2024-04-25T20:11:14.407530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:10.610071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"64'77","reported_seq":169,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T20:11:15.198319+0000","last_change":"2024-04-25T20:11:12.399325+0000","last_active":"2024-04-25T20:11:15.198319+0000","last_peered":"2024-04-25T20:11:15.198319+0000","last_clean":"2024-04-25T20:11:15.198319+0000","last_became_active":"2024-04-25T20:11:12.398945+0000","last_became_peered":"2024-04-25T20:11:12.398945+0000","last_unstale":"2024-04-25T20:11:15.198319+0000","last_undegraded":"2024-04-25T20:11:15.198319+0000","last_fullsized":"2024-04-25T20:11:15.198319+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:54:02.826800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030896500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":77,"num_write":23,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"64'176","reported_seq":651,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T20:11:14.407986+0000","last_change":"2024-04-25T20:10:53.035823+0000","last_active":"2024-04-25T20:11:14.407986+0000","last_peered":"2024-04-25T20:11:14.407986+0000","last_clean":"2024-04-25T20:11:14.407986+0000","last_became_active":"2024-04-25T20:09:16.140731+0000","last_became_peered":"2024-04-25T20:09:16.140731+0000","last_unstale":"2024-04-25T20:11:14.407986+0000","last_undegraded":"2024-04-25T20:11:14.407986+0000","last_fullsized":"2024-04-25T20:11:14.407986+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:18.458296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000234248,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":777,"num_read_kb":681,"num_write":75,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2302,"num_read_kb":2207,"num_write":379,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":1036288,"data_stored":1006490,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48499999999999999,"15min":0.48199999999999998},"min":{"1min":0.32000000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46600000000000003,"15min":0.46700000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.61499999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.46700000000000003,"15min":0.46500000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.77200000000000002}]},{"osd":2,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45700000000000002,"15min":0.45600000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.624,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.624},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.48699999999999999,"15min":0.48299999999999998},"min":{"1min":0.40200000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.66700000000000004}]},{"osd":3,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.75},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.68200000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.68200000000000005}]},{"osd":4,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42699999999999999,"15min":0.42499999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.625,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.374},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.47799999999999998,"15min":0.48199999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.59499999999999997,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.52700000000000002}]},{"osd":5,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48499999999999999,"15min":0.48299999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69099999999999995,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.441}]},{"osd":6,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.373,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68200000000000005,"5min":0.754,"15min":0.754},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66000000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.40699999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32064,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339776,"statfs":{"total":96636764160,"available":96603930624,"internally_reserved":0,"allocated":966656,"data_stored":410614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.503,"15min":0.504},"min":{"1min":0.33400000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72699999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69299999999999995,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.47699999999999998}]},{"osd":1,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.316,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66000000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.42399999999999999}]},{"osd":2,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.501,"15min":0.504},"min":{"1min":0.32000000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.76400000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.503,"5min":0.501,"15min":0.501},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.497}]},{"osd":3,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.5,"15min":0.502},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66200000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65500000000000003,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42799999999999999,"15min":0.42899999999999999},"min":{"1min":0.28299999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64200000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.53000000000000003}]},{"osd":5,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.67900000000000005,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.498,"15min":0.498},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.66300000000000003}]},{"osd":7,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70099999999999996,"5min":0.747,"15min":0.747},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52300000000000002,"15min":0.52200000000000002},"min":{"1min":0.38,"5min":0.37,"15min":0.37},"max":{"1min":0.752,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.622}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":978944,"data_stored":416122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.57999999999999996}]},{"osd":1,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.66500000000000004}]},{"osd":2,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.68500000000000005}]},{"osd":3,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.752},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.56299999999999994}]},{"osd":4,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.63100000000000001}]},{"osd":6,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.72699999999999998}]},{"osd":7,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.69699999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":983040,"data_stored":416141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.58399999999999996}]},{"osd":3,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.504}]},{"osd":5,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.49099999999999999}]},{"osd":6,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.627},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.71499999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":966656,"data_stored":410614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.36799999999999999}]},{"osd":1,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.34899999999999998}]},{"osd":2,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.44}]},{"osd":4,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.61699999999999999}]},{"osd":5,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.69799999999999995}]},{"osd":6,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.629}]},{"osd":7,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.72499999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":978944,"data_stored":416122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.20999999999999999}]},{"osd":1,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.29099999999999998}]},{"osd":3,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.314},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.44500000000000001}]},{"osd":5,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47999999999999998}]},{"osd":6,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.55500000000000005}]},{"osd":7,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.57799999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":1036288,"data_stored":1006490,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":3.0569999999999999,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.221},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":2.9660000000000002,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":0.24199999999999999}]},{"osd":2,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.0289999999999999,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.28899999999999998}]},{"osd":3,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.0859999999999999,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.9950000000000001,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.5}]},{"osd":5,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.62},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.57799999999999996}]},{"osd":6,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54100000000000004}]},{"osd":7,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.64100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":978944,"data_stored":415205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.36099999999999999,"15min":0.35599999999999998},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.38200000000000001,"15min":0.375},"min":{"1min":0.24199999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.432}]},{"osd":2,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.39700000000000002,"15min":0.39100000000000001},"min":{"1min":0.23400000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.40500000000000003,"15min":0.39800000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.36199999999999999}]},{"osd":3,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.36199999999999999,"15min":0.36099999999999999},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.43099999999999999,"15min":0.42399999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65900000000000003,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.38200000000000001}]},{"osd":4,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.79300000000000004,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73799999999999999,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.40300000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79200000000000004,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.42799999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86099999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.51300000000000001}]},{"osd":6,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.879,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81399999999999995,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.60199999999999998}]},{"osd":7,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.622,"15min":0.625},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91100000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84499999999999997,"5min":1.079,"15min":1.079},"last":0.498}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:11:19.001 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T20:11:19.293 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:11:19.294 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-04-25T20:11:19.306 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-04-25T20:11:18.459666+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2418,"num_read_kb":2307,"num_write":538,"num_write_kb":2800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":979,"ondisk_log_size":979,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248512,"kb_used_data":7740,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754726208,"statfs":{"total":773094113280,"available":772839636992,"internally_reserved":0,"allocated":7925760,"data_stored":4497798,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12741,"internal_metadata":246402619},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":-539,"num_read_kb":-490,"num_write":-41,"num_write_kb":-44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.060181"},"pg_stats":[{"pgid":"1.7","version":"64'117","reported_seq":322,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T20:11:15.198251+0000","last_change":"2024-04-25T20:11:03.266483+0000","last_active":"2024-04-25T20:11:15.198251+0000","last_peered":"2024-04-25T20:11:15.198251+0000","last_clean":"2024-04-25T20:11:15.198251+0000","last_became_active":"2024-04-25T20:11:03.266120+0000","last_became_peered":"2024-04-25T20:11:03.266120+0000","last_unstale":"2024-04-25T20:11:15.198251+0000","last_undegraded":"2024-04-25T20:11:15.198251+0000","last_fullsized":"2024-04-25T20:11:15.198251+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:36.703817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033361900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":159,"num_read_kb":209,"num_write":53,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"64'156","reported_seq":500,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T20:11:15.198216+0000","last_change":"2024-04-25T20:10:53.077262+0000","last_active":"2024-04-25T20:11:15.198216+0000","last_peered":"2024-04-25T20:11:15.198216+0000","last_clean":"2024-04-25T20:11:15.198216+0000","last_became_active":"2024-04-25T20:09:16.140616+0000","last_became_peered":"2024-04-25T20:09:16.140616+0000","last_unstale":"2024-04-25T20:11:15.198216+0000","last_undegraded":"2024-04-25T20:11:15.198216+0000","last_fullsized":"2024-04-25T20:11:15.198216+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:31.562175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000351966,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":530,"num_read_kb":465,"num_write":90,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"64'104","reported_seq":419,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T20:11:14.403343+0000","last_change":"2024-04-25T20:11:08.336815+0000","last_active":"2024-04-25T20:11:14.403343+0000","last_peered":"2024-04-25T20:11:14.403343+0000","last_clean":"2024-04-25T20:11:14.403343+0000","last_became_active":"2024-04-25T20:11:08.336485+0000","last_became_peered":"2024-04-25T20:11:08.336485+0000","last_unstale":"2024-04-25T20:11:14.403343+0000","last_undegraded":"2024-04-25T20:11:14.403343+0000","last_fullsized":"2024-04-25T20:11:14.403343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:37.372937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023598099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":408,"num_read_kb":340,"num_write":32,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"64'113","reported_seq":236,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T20:11:14.404154+0000","last_change":"2024-04-25T20:11:03.270954+0000","last_active":"2024-04-25T20:11:14.404154+0000","last_peered":"2024-04-25T20:11:14.404154+0000","last_clean":"2024-04-25T20:11:14.404154+0000","last_became_active":"2024-04-25T20:11:03.270477+0000","last_became_peered":"2024-04-25T20:11:03.270477+0000","last_unstale":"2024-04-25T20:11:14.404154+0000","last_undegraded":"2024-04-25T20:11:14.404154+0000","last_fullsized":"2024-04-25T20:11:14.404154+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:53.502563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024956199999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":99,"num_read_kb":219,"num_write":57,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":222,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T20:11:14.407982+0000","last_change":"2024-04-25T20:09:17.121148+0000","last_active":"2024-04-25T20:11:14.407982+0000","last_peered":"2024-04-25T20:11:14.407982+0000","last_clean":"2024-04-25T20:11:14.407982+0000","last_became_active":"2024-04-25T20:09:17.120312+0000","last_became_peered":"2024-04-25T20:09:17.120312+0000","last_unstale":"2024-04-25T20:11:14.407982+0000","last_undegraded":"2024-04-25T20:11:14.407982+0000","last_fullsized":"2024-04-25T20:11:14.407982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_clean_scrub_stamp":"2024-04-25T20:09:15.745353+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:48.383841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"64'119","reported_seq":323,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T20:11:15.198041+0000","last_change":"2024-04-25T20:10:53.077437+0000","last_active":"2024-04-25T20:11:15.198041+0000","last_peered":"2024-04-25T20:11:15.198041+0000","last_clean":"2024-04-25T20:11:15.198041+0000","last_became_active":"2024-04-25T20:09:15.772551+0000","last_became_peered":"2024-04-25T20:09:15.772551+0000","last_unstale":"2024-04-25T20:11:15.198041+0000","last_undegraded":"2024-04-25T20:11:15.198041+0000","last_fullsized":"2024-04-25T20:11:15.198041+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:55.101025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035253999999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":240,"num_read_kb":216,"num_write":49,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"unknown","last_fresh":"2024-04-25T20:11:14.407530+0000","last_change":"2024-04-25T20:11:14.647838+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T20:11:14.407530+0000","last_undegraded":"2024-04-25T20:11:14.407530+0000","last_fullsized":"2024-04-25T20:11:14.407530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:10.610071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"64'77","reported_seq":169,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T20:11:15.198319+0000","last_change":"2024-04-25T20:11:12.399325+0000","last_active":"2024-04-25T20:11:15.198319+0000","last_peered":"2024-04-25T20:11:15.198319+0000","last_clean":"2024-04-25T20:11:15.198319+0000","last_became_active":"2024-04-25T20:11:12.398945+0000","last_became_peered":"2024-04-25T20:11:12.398945+0000","last_unstale":"2024-04-25T20:11:15.198319+0000","last_undegraded":"2024-04-25T20:11:15.198319+0000","last_fullsized":"2024-04-25T20:11:15.198319+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:54:02.826800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030896500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":77,"num_write":23,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"64'176","reported_seq":651,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T20:11:14.407986+0000","last_change":"2024-04-25T20:10:53.035823+0000","last_active":"2024-04-25T20:11:14.407986+0000","last_peered":"2024-04-25T20:11:14.407986+0000","last_clean":"2024-04-25T20:11:14.407986+0000","last_became_active":"2024-04-25T20:09:16.140731+0000","last_became_peered":"2024-04-25T20:09:16.140731+0000","last_unstale":"2024-04-25T20:11:14.407986+0000","last_undegraded":"2024-04-25T20:11:14.407986+0000","last_fullsized":"2024-04-25T20:11:14.407986+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:18.458296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000234248,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":777,"num_read_kb":681,"num_write":75,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2302,"num_read_kb":2207,"num_write":379,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":1036288,"data_stored":1006490,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48499999999999999,"15min":0.48199999999999998},"min":{"1min":0.32000000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46600000000000003,"15min":0.46700000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.61499999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.46700000000000003,"15min":0.46500000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.77200000000000002}]},{"osd":2,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45700000000000002,"15min":0.45600000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.624,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.624},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.48699999999999999,"15min":0.48299999999999998},"min":{"1min":0.40200000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.66700000000000004}]},{"osd":3,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.75},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.68200000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.68200000000000005}]},{"osd":4,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42699999999999999,"15min":0.42499999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.625,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.374},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.47799999999999998,"15min":0.48199999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.59499999999999997,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.52700000000000002}]},{"osd":5,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48499999999999999,"15min":0.48299999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69099999999999995,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.441}]},{"osd":6,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.373,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68200000000000005,"5min":0.754,"15min":0.754},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66000000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.40699999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32064,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339776,"statfs":{"total":96636764160,"available":96603930624,"internally_reserved":0,"allocated":966656,"data_stored":410614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.503,"15min":0.504},"min":{"1min":0.33400000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72699999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69299999999999995,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.47699999999999998}]},{"osd":1,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.316,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66000000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.42399999999999999}]},{"osd":2,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.501,"15min":0.504},"min":{"1min":0.32000000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.76400000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.503,"5min":0.501,"15min":0.501},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.497}]},{"osd":3,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.5,"15min":0.502},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66200000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65500000000000003,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42799999999999999,"15min":0.42899999999999999},"min":{"1min":0.28299999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64200000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.53000000000000003}]},{"osd":5,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.67900000000000005,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.498,"15min":0.498},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.66300000000000003}]},{"osd":7,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70099999999999996,"5min":0.747,"15min":0.747},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52300000000000002,"15min":0.52200000000000002},"min":{"1min":0.38,"5min":0.37,"15min":0.37},"max":{"1min":0.752,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.622}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":978944,"data_stored":416122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.57999999999999996}]},{"osd":1,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.66500000000000004}]},{"osd":2,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.68500000000000005}]},{"osd":3,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.752},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.56299999999999994}]},{"osd":4,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.63100000000000001}]},{"osd":6,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.72699999999999998}]},{"osd":7,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.69699999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":983040,"data_stored":416141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.58399999999999996}]},{"osd":3,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.504}]},{"osd":5,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.49099999999999999}]},{"osd":6,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.627},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.71499999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":966656,"data_stored":410614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.36799999999999999}]},{"osd":1,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.34899999999999998}]},{"osd":2,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.44}]},{"osd":4,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.61699999999999999}]},{"osd":5,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.69799999999999995}]},{"osd":6,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.629}]},{"osd":7,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.72499999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":978944,"data_stored":416122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.20999999999999999}]},{"osd":1,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.29099999999999998}]},{"osd":3,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.314},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.44500000000000001}]},{"osd":5,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47999999999999998}]},{"osd":6,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.55500000000000005}]},{"osd":7,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.57799999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":1036288,"data_stored":1006490,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":3.0569999999999999,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.221},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":2.9660000000000002,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":0.24199999999999999}]},{"osd":2,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.0289999999999999,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.28899999999999998}]},{"osd":3,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.0859999999999999,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.9950000000000001,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.5}]},{"osd":5,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.62},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.57799999999999996}]},{"osd":6,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54100000000000004}]},{"osd":7,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.64100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":978944,"data_stored":415205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.36099999999999999,"15min":0.35599999999999998},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.38200000000000001,"15min":0.375},"min":{"1min":0.24199999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.432}]},{"osd":2,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.39700000000000002,"15min":0.39100000000000001},"min":{"1min":0.23400000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.40500000000000003,"15min":0.39800000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.36199999999999999}]},{"osd":3,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.36199999999999999,"15min":0.36099999999999999},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.43099999999999999,"15min":0.42399999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65900000000000003,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.38200000000000001}]},{"osd":4,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.79300000000000004,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73799999999999999,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.40300000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79200000000000004,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.42799999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86099999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.51300000000000001}]},{"osd":6,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.879,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81399999999999995,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.60199999999999998}]},{"osd":7,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.622,"15min":0.625},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91100000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84499999999999997,"5min":1.079,"15min":1.079},"last":0.498}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:11:19.308 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T20:11:19.666 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:11:19.666 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"975ceafd-555d-4f71-81e4-1f70364da00f","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":136,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":86,"num_osds":8,"num_up_osds":8,"osd_up_since":1714075753,"num_in_osds":8,"osd_in_since":1714075859,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":8},{"state_name":"unknown","count":1}],"num_pgs":9,"num_pools":2,"num_objects":4,"data_bytes":590368,"bytes_used":254476288,"bytes_avail":772839636992,"bytes_total":773094113280,"unknown_pgs_ratio":0.1111111119389534},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-04-25T20:10:15.611469+0000","services":{}},"progress_events":{}} 2024-04-25T20:11:19.679 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-04-25T20:11:19.679 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T20:11:19.967 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:11:19.967 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-04-25T20:11:19.979 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-04-25T20:11:19.621191+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2418,"num_read_kb":2307,"num_write":538,"num_write_kb":2800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":979,"ondisk_log_size":979,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248512,"kb_used_data":7740,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754726208,"statfs":{"total":773094113280,"available":772839636992,"internally_reserved":0,"allocated":7925760,"data_stored":4497798,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12741,"internal_metadata":246402619},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.202833"},"pg_stats":[{"pgid":"1.7","version":"64'117","reported_seq":322,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T20:11:15.198251+0000","last_change":"2024-04-25T20:11:03.266483+0000","last_active":"2024-04-25T20:11:15.198251+0000","last_peered":"2024-04-25T20:11:15.198251+0000","last_clean":"2024-04-25T20:11:15.198251+0000","last_became_active":"2024-04-25T20:11:03.266120+0000","last_became_peered":"2024-04-25T20:11:03.266120+0000","last_unstale":"2024-04-25T20:11:15.198251+0000","last_undegraded":"2024-04-25T20:11:15.198251+0000","last_fullsized":"2024-04-25T20:11:15.198251+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:36.703817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033361900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":159,"num_read_kb":209,"num_write":53,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"64'156","reported_seq":500,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T20:11:15.198216+0000","last_change":"2024-04-25T20:10:53.077262+0000","last_active":"2024-04-25T20:11:15.198216+0000","last_peered":"2024-04-25T20:11:15.198216+0000","last_clean":"2024-04-25T20:11:15.198216+0000","last_became_active":"2024-04-25T20:09:16.140616+0000","last_became_peered":"2024-04-25T20:09:16.140616+0000","last_unstale":"2024-04-25T20:11:15.198216+0000","last_undegraded":"2024-04-25T20:11:15.198216+0000","last_fullsized":"2024-04-25T20:11:15.198216+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:31.562175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000351966,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":530,"num_read_kb":465,"num_write":90,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"64'104","reported_seq":419,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T20:11:14.403343+0000","last_change":"2024-04-25T20:11:08.336815+0000","last_active":"2024-04-25T20:11:14.403343+0000","last_peered":"2024-04-25T20:11:14.403343+0000","last_clean":"2024-04-25T20:11:14.403343+0000","last_became_active":"2024-04-25T20:11:08.336485+0000","last_became_peered":"2024-04-25T20:11:08.336485+0000","last_unstale":"2024-04-25T20:11:14.403343+0000","last_undegraded":"2024-04-25T20:11:14.403343+0000","last_fullsized":"2024-04-25T20:11:14.403343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:37.372937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023598099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":408,"num_read_kb":340,"num_write":32,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"64'113","reported_seq":236,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T20:11:14.404154+0000","last_change":"2024-04-25T20:11:03.270954+0000","last_active":"2024-04-25T20:11:14.404154+0000","last_peered":"2024-04-25T20:11:14.404154+0000","last_clean":"2024-04-25T20:11:14.404154+0000","last_became_active":"2024-04-25T20:11:03.270477+0000","last_became_peered":"2024-04-25T20:11:03.270477+0000","last_unstale":"2024-04-25T20:11:14.404154+0000","last_undegraded":"2024-04-25T20:11:14.404154+0000","last_fullsized":"2024-04-25T20:11:14.404154+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:53.502563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024956199999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":99,"num_read_kb":219,"num_write":57,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":222,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T20:11:14.407982+0000","last_change":"2024-04-25T20:09:17.121148+0000","last_active":"2024-04-25T20:11:14.407982+0000","last_peered":"2024-04-25T20:11:14.407982+0000","last_clean":"2024-04-25T20:11:14.407982+0000","last_became_active":"2024-04-25T20:09:17.120312+0000","last_became_peered":"2024-04-25T20:09:17.120312+0000","last_unstale":"2024-04-25T20:11:14.407982+0000","last_undegraded":"2024-04-25T20:11:14.407982+0000","last_fullsized":"2024-04-25T20:11:14.407982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_clean_scrub_stamp":"2024-04-25T20:09:15.745353+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:48.383841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"64'119","reported_seq":323,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T20:11:15.198041+0000","last_change":"2024-04-25T20:10:53.077437+0000","last_active":"2024-04-25T20:11:15.198041+0000","last_peered":"2024-04-25T20:11:15.198041+0000","last_clean":"2024-04-25T20:11:15.198041+0000","last_became_active":"2024-04-25T20:09:15.772551+0000","last_became_peered":"2024-04-25T20:09:15.772551+0000","last_unstale":"2024-04-25T20:11:15.198041+0000","last_undegraded":"2024-04-25T20:11:15.198041+0000","last_fullsized":"2024-04-25T20:11:15.198041+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:55.101025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035253999999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":240,"num_read_kb":216,"num_write":49,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"unknown","last_fresh":"2024-04-25T20:11:14.407530+0000","last_change":"2024-04-25T20:11:14.647838+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T20:11:14.407530+0000","last_undegraded":"2024-04-25T20:11:14.407530+0000","last_fullsized":"2024-04-25T20:11:14.407530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:10.610071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"64'77","reported_seq":169,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T20:11:15.198319+0000","last_change":"2024-04-25T20:11:12.399325+0000","last_active":"2024-04-25T20:11:15.198319+0000","last_peered":"2024-04-25T20:11:15.198319+0000","last_clean":"2024-04-25T20:11:15.198319+0000","last_became_active":"2024-04-25T20:11:12.398945+0000","last_became_peered":"2024-04-25T20:11:12.398945+0000","last_unstale":"2024-04-25T20:11:15.198319+0000","last_undegraded":"2024-04-25T20:11:15.198319+0000","last_fullsized":"2024-04-25T20:11:15.198319+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:54:02.826800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030896500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":77,"num_write":23,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"64'176","reported_seq":651,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T20:11:14.407986+0000","last_change":"2024-04-25T20:10:53.035823+0000","last_active":"2024-04-25T20:11:14.407986+0000","last_peered":"2024-04-25T20:11:14.407986+0000","last_clean":"2024-04-25T20:11:14.407986+0000","last_became_active":"2024-04-25T20:09:16.140731+0000","last_became_peered":"2024-04-25T20:09:16.140731+0000","last_unstale":"2024-04-25T20:11:14.407986+0000","last_undegraded":"2024-04-25T20:11:14.407986+0000","last_fullsized":"2024-04-25T20:11:14.407986+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:18.458296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000234248,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":777,"num_read_kb":681,"num_write":75,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2302,"num_read_kb":2207,"num_write":379,"num_write_kb":384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":1036288,"data_stored":1006490,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48499999999999999,"15min":0.48199999999999998},"min":{"1min":0.32000000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46600000000000003,"15min":0.46700000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.61499999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.46700000000000003,"15min":0.46500000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.77200000000000002}]},{"osd":2,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45700000000000002,"15min":0.45600000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.624,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.624},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.48699999999999999,"15min":0.48299999999999998},"min":{"1min":0.40200000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.66700000000000004}]},{"osd":3,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.75},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.68200000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.68200000000000005}]},{"osd":4,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42699999999999999,"15min":0.42499999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.625,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.374},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.47799999999999998,"15min":0.48199999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.59499999999999997,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.52700000000000002}]},{"osd":5,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48499999999999999,"15min":0.48299999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69099999999999995,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.441}]},{"osd":6,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.373,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68200000000000005,"5min":0.754,"15min":0.754},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66000000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.40699999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32064,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339776,"statfs":{"total":96636764160,"available":96603930624,"internally_reserved":0,"allocated":966656,"data_stored":410614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.503,"15min":0.504},"min":{"1min":0.33400000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72699999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69299999999999995,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.47699999999999998}]},{"osd":1,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.316,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66000000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.42399999999999999}]},{"osd":2,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.501,"15min":0.504},"min":{"1min":0.32000000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.76400000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.503,"5min":0.501,"15min":0.501},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.497}]},{"osd":3,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.5,"15min":0.502},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66200000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65500000000000003,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42799999999999999,"15min":0.42899999999999999},"min":{"1min":0.28299999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64200000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.53000000000000003}]},{"osd":5,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.67900000000000005,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.498,"15min":0.498},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.66300000000000003}]},{"osd":7,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70099999999999996,"5min":0.747,"15min":0.747},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52300000000000002,"15min":0.52200000000000002},"min":{"1min":0.38,"5min":0.37,"15min":0.37},"max":{"1min":0.752,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.622}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":978944,"data_stored":416122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.57999999999999996}]},{"osd":1,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.66500000000000004}]},{"osd":2,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.68500000000000005}]},{"osd":3,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.752},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.56299999999999994}]},{"osd":4,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.63100000000000001}]},{"osd":6,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.72699999999999998}]},{"osd":7,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.69699999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":983040,"data_stored":416141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.58399999999999996}]},{"osd":3,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.504}]},{"osd":5,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.49099999999999999}]},{"osd":6,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.627},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.71499999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":966656,"data_stored":410614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.36799999999999999}]},{"osd":1,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.34899999999999998}]},{"osd":2,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.44}]},{"osd":4,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.61699999999999999}]},{"osd":5,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.69799999999999995}]},{"osd":6,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.629}]},{"osd":7,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.72499999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":978944,"data_stored":416122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.20999999999999999}]},{"osd":1,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.29099999999999998}]},{"osd":3,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.314},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.44500000000000001}]},{"osd":5,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47999999999999998}]},{"osd":6,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.55500000000000005}]},{"osd":7,"last update":"Thu Apr 25 20:10:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.57799999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":1036288,"data_stored":1006490,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":3.0569999999999999,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.221},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":2.9660000000000002,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":0.24199999999999999}]},{"osd":2,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.0289999999999999,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.28899999999999998}]},{"osd":3,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.0859999999999999,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.9950000000000001,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.5}]},{"osd":5,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.62},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.57799999999999996}]},{"osd":6,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54100000000000004}]},{"osd":7,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.64100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":978944,"data_stored":415205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.36099999999999999,"15min":0.35599999999999998},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.38200000000000001,"15min":0.375},"min":{"1min":0.24199999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.432}]},{"osd":2,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.39700000000000002,"15min":0.39100000000000001},"min":{"1min":0.23400000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.40500000000000003,"15min":0.39800000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.36199999999999999}]},{"osd":3,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.36199999999999999,"15min":0.36099999999999999},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.43099999999999999,"15min":0.42399999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65900000000000003,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.38200000000000001}]},{"osd":4,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.79300000000000004,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73799999999999999,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.40300000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79200000000000004,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.42799999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86099999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.51300000000000001}]},{"osd":6,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.879,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81399999999999995,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.60199999999999998}]},{"osd":7,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.622,"15min":0.625},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91100000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84499999999999997,"5min":1.079,"15min":1.079},"last":0.498}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:11:22.982 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T20:11:23.287 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:11:23.288 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-04-25T20:11:23.300 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-04-25T20:11:21.621584+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2957,"num_read_kb":2797,"num_write":579,"num_write_kb":2844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1096,"ondisk_log_size":1096,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248756,"kb_used_data":7992,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754725964,"statfs":{"total":773094113280,"available":772839387136,"internally_reserved":0,"allocated":8183808,"data_stored":4612732,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12741,"internal_metadata":246402619},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":539,"num_read_kb":490,"num_write":41,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000950"},"pg_stats":[{"pgid":"1.7","version":"64'117","reported_seq":324,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T20:11:16.801654+0000","last_change":"2024-04-25T20:11:03.266483+0000","last_active":"2024-04-25T20:11:16.801654+0000","last_peered":"2024-04-25T20:11:16.801654+0000","last_clean":"2024-04-25T20:11:16.801654+0000","last_became_active":"2024-04-25T20:11:03.266120+0000","last_became_peered":"2024-04-25T20:11:03.266120+0000","last_unstale":"2024-04-25T20:11:16.801654+0000","last_undegraded":"2024-04-25T20:11:16.801654+0000","last_fullsized":"2024-04-25T20:11:16.801654+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:36.703817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033361900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":159,"num_read_kb":209,"num_write":53,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"64'156","reported_seq":502,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T20:11:16.801650+0000","last_change":"2024-04-25T20:10:53.077262+0000","last_active":"2024-04-25T20:11:16.801650+0000","last_peered":"2024-04-25T20:11:16.801650+0000","last_clean":"2024-04-25T20:11:16.801650+0000","last_became_active":"2024-04-25T20:09:16.140616+0000","last_became_peered":"2024-04-25T20:09:16.140616+0000","last_unstale":"2024-04-25T20:11:16.801650+0000","last_undegraded":"2024-04-25T20:11:16.801650+0000","last_fullsized":"2024-04-25T20:11:16.801650+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:31.562175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000351966,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":530,"num_read_kb":465,"num_write":90,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"64'104","reported_seq":422,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T20:11:17.446087+0000","last_change":"2024-04-25T20:11:08.336815+0000","last_active":"2024-04-25T20:11:17.446087+0000","last_peered":"2024-04-25T20:11:17.446087+0000","last_clean":"2024-04-25T20:11:17.446087+0000","last_became_active":"2024-04-25T20:11:08.336485+0000","last_became_peered":"2024-04-25T20:11:08.336485+0000","last_unstale":"2024-04-25T20:11:17.446087+0000","last_undegraded":"2024-04-25T20:11:17.446087+0000","last_fullsized":"2024-04-25T20:11:17.446087+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:37.372937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023598099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":408,"num_read_kb":340,"num_write":32,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"64'113","reported_seq":239,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T20:11:18.038265+0000","last_change":"2024-04-25T20:11:03.270954+0000","last_active":"2024-04-25T20:11:18.038265+0000","last_peered":"2024-04-25T20:11:18.038265+0000","last_clean":"2024-04-25T20:11:18.038265+0000","last_became_active":"2024-04-25T20:11:03.270477+0000","last_became_peered":"2024-04-25T20:11:03.270477+0000","last_unstale":"2024-04-25T20:11:18.038265+0000","last_undegraded":"2024-04-25T20:11:18.038265+0000","last_fullsized":"2024-04-25T20:11:18.038265+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:53.502563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024956199999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":99,"num_read_kb":219,"num_write":57,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":226,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T20:11:18.458187+0000","last_change":"2024-04-25T20:09:17.121148+0000","last_active":"2024-04-25T20:11:18.458187+0000","last_peered":"2024-04-25T20:11:18.458187+0000","last_clean":"2024-04-25T20:11:18.458187+0000","last_became_active":"2024-04-25T20:09:17.120312+0000","last_became_peered":"2024-04-25T20:09:17.120312+0000","last_unstale":"2024-04-25T20:11:18.458187+0000","last_undegraded":"2024-04-25T20:11:18.458187+0000","last_fullsized":"2024-04-25T20:11:18.458187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_clean_scrub_stamp":"2024-04-25T20:09:15.745353+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:48.383841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"64'119","reported_seq":325,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T20:11:16.430739+0000","last_change":"2024-04-25T20:10:53.077437+0000","last_active":"2024-04-25T20:11:16.430739+0000","last_peered":"2024-04-25T20:11:16.430739+0000","last_clean":"2024-04-25T20:11:16.430739+0000","last_became_active":"2024-04-25T20:09:15.772551+0000","last_became_peered":"2024-04-25T20:09:15.772551+0000","last_unstale":"2024-04-25T20:11:16.430739+0000","last_undegraded":"2024-04-25T20:11:16.430739+0000","last_fullsized":"2024-04-25T20:11:16.430739+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:55.101025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035253999999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":240,"num_read_kb":216,"num_write":49,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"64'117","reported_seq":436,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T20:11:18.462309+0000","last_change":"2024-04-25T20:11:18.462309+0000","last_active":"2024-04-25T20:11:18.462309+0000","last_peered":"2024-04-25T20:11:18.462309+0000","last_clean":"2024-04-25T20:11:18.462309+0000","last_became_active":"2024-04-25T20:11:18.461739+0000","last_became_peered":"2024-04-25T20:11:18.461739+0000","last_unstale":"2024-04-25T20:11:18.462309+0000","last_undegraded":"2024-04-25T20:11:18.462309+0000","last_fullsized":"2024-04-25T20:11:18.462309+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:48:03.001662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001291134,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":539,"num_read_kb":490,"num_write":41,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"64'77","reported_seq":172,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T20:11:17.442232+0000","last_change":"2024-04-25T20:11:12.399325+0000","last_active":"2024-04-25T20:11:17.442232+0000","last_peered":"2024-04-25T20:11:17.442232+0000","last_clean":"2024-04-25T20:11:17.442232+0000","last_became_active":"2024-04-25T20:11:12.398945+0000","last_became_peered":"2024-04-25T20:11:12.398945+0000","last_unstale":"2024-04-25T20:11:17.442232+0000","last_undegraded":"2024-04-25T20:11:17.442232+0000","last_fullsized":"2024-04-25T20:11:17.442232+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:54:02.826800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030896500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":77,"num_write":23,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"64'176","reported_seq":655,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T20:11:18.458187+0000","last_change":"2024-04-25T20:10:53.035823+0000","last_active":"2024-04-25T20:11:18.458187+0000","last_peered":"2024-04-25T20:11:18.458187+0000","last_clean":"2024-04-25T20:11:18.458187+0000","last_became_active":"2024-04-25T20:09:16.140731+0000","last_became_peered":"2024-04-25T20:09:16.140731+0000","last_unstale":"2024-04-25T20:11:18.458187+0000","last_undegraded":"2024-04-25T20:11:18.458187+0000","last_fullsized":"2024-04-25T20:11:18.458187+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:18.458296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000234248,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":777,"num_read_kb":681,"num_write":75,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2841,"num_read_kb":2697,"num_write":420,"num_write_kb":428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":979,"ondisk_log_size":979,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1077248,"data_stored":1022905,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48499999999999999,"15min":0.48199999999999998},"min":{"1min":0.32000000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46600000000000003,"15min":0.46700000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.61499999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.46700000000000003,"15min":0.46500000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.77200000000000002}]},{"osd":2,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45700000000000002,"15min":0.45600000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.624,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.624},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.48699999999999999,"15min":0.48299999999999998},"min":{"1min":0.40200000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.66700000000000004}]},{"osd":3,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.75},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.68200000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.68200000000000005}]},{"osd":4,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42699999999999999,"15min":0.42499999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.625,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.374},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.47799999999999998,"15min":0.48199999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.59499999999999997,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.52700000000000002}]},{"osd":5,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48499999999999999,"15min":0.48299999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69099999999999995,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.441}]},{"osd":6,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.373,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68200000000000005,"5min":0.754,"15min":0.754},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66000000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.40699999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":1003520,"data_stored":427068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.503,"15min":0.504},"min":{"1min":0.33400000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72699999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69299999999999995,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.316,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66000000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.46800000000000003}]},{"osd":2,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.501,"15min":0.504},"min":{"1min":0.32000000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.76400000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.501,"15min":0.501},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.65800000000000003}]},{"osd":3,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.5,"15min":0.502},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66200000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65500000000000003,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.60199999999999998}]},{"osd":4,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42799999999999999,"15min":0.42899999999999999},"min":{"1min":0.28299999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64200000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.63500000000000001}]},{"osd":5,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.67900000000000005,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.498,"15min":0.498},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.71499999999999997}]},{"osd":7,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70099999999999996,"5min":0.747,"15min":0.747},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52300000000000002,"15min":0.52200000000000002},"min":{"1min":0.38,"5min":0.37,"15min":0.37},"max":{"1min":0.752,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.82299999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1003520,"data_stored":427068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.65200000000000002}]},{"osd":1,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.433},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.54300000000000004}]},{"osd":2,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.85699999999999998}]},{"osd":3,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.90600000000000003}]},{"osd":4,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.5},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.72099999999999997}]},{"osd":6,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.69099999999999995}]},{"osd":7,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.83999999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1015808,"data_stored":432518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.58399999999999996}]},{"osd":3,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.504}]},{"osd":5,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.49099999999999999}]},{"osd":6,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.627},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.71499999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":1003520,"data_stored":427068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.37,"15min":0.35599999999999998},"min":{"1min":0.247,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.379,"15min":0.35499999999999998},"min":{"1min":0.255,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.35399999999999998}]},{"osd":1,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.38700000000000001,"15min":0.371},"min":{"1min":0.316,"5min":0.223,"15min":0.223},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.41999999999999998,"15min":0.40799999999999997},"min":{"1min":0.30099999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.311}]},{"osd":2,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.42099999999999999,"15min":0.40100000000000002},"min":{"1min":0.31,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.42199999999999999,"15min":0.40400000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.33400000000000002}]},{"osd":4,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60699999999999998,"15min":0.59499999999999997},"min":{"1min":0.48199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1160000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.59799999999999998,"15min":0.58699999999999997},"min":{"1min":0.44900000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93799999999999994,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.68600000000000005}]},{"osd":5,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.41999999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.044,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.46200000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.497}]},{"osd":6,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.61499999999999999,"15min":0.60099999999999998},"min":{"1min":0.43099999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.59999999999999998,"15min":0.58199999999999996},"min":{"1min":0.48599999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1459999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.51500000000000001}]},{"osd":7,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.51100000000000001,"5min":0.37,"15min":0.37},"max":{"1min":1.171,"5min":1.272,"15min":1.272},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.63500000000000001,"15min":0.624},"min":{"1min":0.47499999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.1020000000000001,"5min":1.302,"15min":1.302},"last":0.71599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":1003520,"data_stored":427068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.36399999999999999,"15min":0.35799999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.34699999999999998,"15min":0.33600000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.255}]},{"osd":1,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.33900000000000002,"15min":0.33300000000000002},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.40100000000000002,"15min":0.39700000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.317}]},{"osd":3,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.39900000000000002,"15min":0.39300000000000002},"min":{"1min":0.27500000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.432,"15min":0.42699999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.378}]},{"osd":4,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53500000000000003,"15min":0.52700000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85199999999999998,"5min":1.304,"15min":1.304},"last":0.56299999999999994}]},{"osd":5,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.73099999999999998,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.78500000000000003}]},{"osd":6,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.438,"5min":0.433,"15min":0.433},"max":{"1min":0.88900000000000001,"5min":1.339,"15min":1.339},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.58599999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77800000000000002,"5min":0.872,"15min":0.872},"last":0.58899999999999997}]},{"osd":7,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.44700000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.2829999999999999,"5min":1.321,"15min":1.321},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81599999999999995,"5min":1.458,"15min":1.458},"last":0.76300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":1060864,"data_stored":1017436,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":3.0569999999999999,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.221},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":2.9660000000000002,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":0.24199999999999999}]},{"osd":2,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.0289999999999999,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.28899999999999998}]},{"osd":3,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.0859999999999999,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.9950000000000001,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.5}]},{"osd":5,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.62},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.57799999999999996}]},{"osd":6,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54100000000000004}]},{"osd":7,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.64100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":1015808,"data_stored":431601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.36099999999999999,"15min":0.35599999999999998},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.38200000000000001,"15min":0.375},"min":{"1min":0.24199999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":1.016}]},{"osd":2,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.39700000000000002,"15min":0.39100000000000001},"min":{"1min":0.23400000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":1.141},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.40500000000000003,"15min":0.39800000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":1.3049999999999999}]},{"osd":3,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.36199999999999999,"15min":0.36099999999999999},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.751},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.43099999999999999,"15min":0.42399999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65900000000000003,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.995}]},{"osd":4,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.79300000000000004,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73799999999999999,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.67700000000000005}]},{"osd":5,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.40300000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79200000000000004,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.42799999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86099999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.51300000000000001}]},{"osd":6,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.879,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81399999999999995,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.92900000000000005}]},{"osd":7,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.622,"15min":0.625},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91100000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84499999999999997,"5min":1.079,"15min":1.079},"last":0.57299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:11:23.302 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-25T20:11:23.302 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-04-25T20:11:23.614 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T20:11:23.623 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T20:11:23.623 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T20:11:23.918 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:11:23.919 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-04-25T20:11:23.931 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-04-25T20:11:23.621971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2957,"num_read_kb":2797,"num_write":579,"num_write_kb":2844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1096,"ondisk_log_size":1096,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248756,"kb_used_data":7992,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754725964,"statfs":{"total":773094113280,"available":772839387136,"internally_reserved":0,"allocated":8183808,"data_stored":4612732,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12741,"internal_metadata":246402619},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":539,"num_read_kb":490,"num_write":41,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.190589"},"pg_stats":[{"pgid":"1.7","version":"64'117","reported_seq":324,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T20:11:16.801654+0000","last_change":"2024-04-25T20:11:03.266483+0000","last_active":"2024-04-25T20:11:16.801654+0000","last_peered":"2024-04-25T20:11:16.801654+0000","last_clean":"2024-04-25T20:11:16.801654+0000","last_became_active":"2024-04-25T20:11:03.266120+0000","last_became_peered":"2024-04-25T20:11:03.266120+0000","last_unstale":"2024-04-25T20:11:16.801654+0000","last_undegraded":"2024-04-25T20:11:16.801654+0000","last_fullsized":"2024-04-25T20:11:16.801654+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:36.703817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033361900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":159,"num_read_kb":209,"num_write":53,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"64'156","reported_seq":502,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T20:11:16.801650+0000","last_change":"2024-04-25T20:10:53.077262+0000","last_active":"2024-04-25T20:11:16.801650+0000","last_peered":"2024-04-25T20:11:16.801650+0000","last_clean":"2024-04-25T20:11:16.801650+0000","last_became_active":"2024-04-25T20:09:16.140616+0000","last_became_peered":"2024-04-25T20:09:16.140616+0000","last_unstale":"2024-04-25T20:11:16.801650+0000","last_undegraded":"2024-04-25T20:11:16.801650+0000","last_fullsized":"2024-04-25T20:11:16.801650+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:31.562175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000351966,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":530,"num_read_kb":465,"num_write":90,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"64'104","reported_seq":422,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T20:11:17.446087+0000","last_change":"2024-04-25T20:11:08.336815+0000","last_active":"2024-04-25T20:11:17.446087+0000","last_peered":"2024-04-25T20:11:17.446087+0000","last_clean":"2024-04-25T20:11:17.446087+0000","last_became_active":"2024-04-25T20:11:08.336485+0000","last_became_peered":"2024-04-25T20:11:08.336485+0000","last_unstale":"2024-04-25T20:11:17.446087+0000","last_undegraded":"2024-04-25T20:11:17.446087+0000","last_fullsized":"2024-04-25T20:11:17.446087+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:37.372937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023598099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":408,"num_read_kb":340,"num_write":32,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"64'113","reported_seq":239,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T20:11:18.038265+0000","last_change":"2024-04-25T20:11:03.270954+0000","last_active":"2024-04-25T20:11:18.038265+0000","last_peered":"2024-04-25T20:11:18.038265+0000","last_clean":"2024-04-25T20:11:18.038265+0000","last_became_active":"2024-04-25T20:11:03.270477+0000","last_became_peered":"2024-04-25T20:11:03.270477+0000","last_unstale":"2024-04-25T20:11:18.038265+0000","last_undegraded":"2024-04-25T20:11:18.038265+0000","last_fullsized":"2024-04-25T20:11:18.038265+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:53.502563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024956199999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":99,"num_read_kb":219,"num_write":57,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":226,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T20:11:18.458187+0000","last_change":"2024-04-25T20:09:17.121148+0000","last_active":"2024-04-25T20:11:18.458187+0000","last_peered":"2024-04-25T20:11:18.458187+0000","last_clean":"2024-04-25T20:11:18.458187+0000","last_became_active":"2024-04-25T20:09:17.120312+0000","last_became_peered":"2024-04-25T20:09:17.120312+0000","last_unstale":"2024-04-25T20:11:18.458187+0000","last_undegraded":"2024-04-25T20:11:18.458187+0000","last_fullsized":"2024-04-25T20:11:18.458187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_clean_scrub_stamp":"2024-04-25T20:09:15.745353+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:48.383841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"64'119","reported_seq":325,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T20:11:16.430739+0000","last_change":"2024-04-25T20:10:53.077437+0000","last_active":"2024-04-25T20:11:16.430739+0000","last_peered":"2024-04-25T20:11:16.430739+0000","last_clean":"2024-04-25T20:11:16.430739+0000","last_became_active":"2024-04-25T20:09:15.772551+0000","last_became_peered":"2024-04-25T20:09:15.772551+0000","last_unstale":"2024-04-25T20:11:16.430739+0000","last_undegraded":"2024-04-25T20:11:16.430739+0000","last_fullsized":"2024-04-25T20:11:16.430739+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:55.101025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035253999999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":240,"num_read_kb":216,"num_write":49,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"64'117","reported_seq":436,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T20:11:18.462309+0000","last_change":"2024-04-25T20:11:18.462309+0000","last_active":"2024-04-25T20:11:18.462309+0000","last_peered":"2024-04-25T20:11:18.462309+0000","last_clean":"2024-04-25T20:11:18.462309+0000","last_became_active":"2024-04-25T20:11:18.461739+0000","last_became_peered":"2024-04-25T20:11:18.461739+0000","last_unstale":"2024-04-25T20:11:18.462309+0000","last_undegraded":"2024-04-25T20:11:18.462309+0000","last_fullsized":"2024-04-25T20:11:18.462309+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:48:03.001662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001291134,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":539,"num_read_kb":490,"num_write":41,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"64'77","reported_seq":172,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T20:11:17.442232+0000","last_change":"2024-04-25T20:11:12.399325+0000","last_active":"2024-04-25T20:11:17.442232+0000","last_peered":"2024-04-25T20:11:17.442232+0000","last_clean":"2024-04-25T20:11:17.442232+0000","last_became_active":"2024-04-25T20:11:12.398945+0000","last_became_peered":"2024-04-25T20:11:12.398945+0000","last_unstale":"2024-04-25T20:11:17.442232+0000","last_undegraded":"2024-04-25T20:11:17.442232+0000","last_fullsized":"2024-04-25T20:11:17.442232+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:54:02.826800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030896500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":77,"num_write":23,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"64'176","reported_seq":655,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T20:11:18.458187+0000","last_change":"2024-04-25T20:10:53.035823+0000","last_active":"2024-04-25T20:11:18.458187+0000","last_peered":"2024-04-25T20:11:18.458187+0000","last_clean":"2024-04-25T20:11:18.458187+0000","last_became_active":"2024-04-25T20:09:16.140731+0000","last_became_peered":"2024-04-25T20:09:16.140731+0000","last_unstale":"2024-04-25T20:11:18.458187+0000","last_undegraded":"2024-04-25T20:11:18.458187+0000","last_fullsized":"2024-04-25T20:11:18.458187+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:18.458296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000234248,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":777,"num_read_kb":681,"num_write":75,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2841,"num_read_kb":2697,"num_write":420,"num_write_kb":428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":979,"ondisk_log_size":979,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1077248,"data_stored":1022905,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48499999999999999,"15min":0.48199999999999998},"min":{"1min":0.32000000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46600000000000003,"15min":0.46700000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.61499999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.46700000000000003,"15min":0.46500000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.77200000000000002}]},{"osd":2,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45700000000000002,"15min":0.45600000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.624,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.624},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.48699999999999999,"15min":0.48299999999999998},"min":{"1min":0.40200000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.66700000000000004}]},{"osd":3,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.75},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.68200000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.68200000000000005}]},{"osd":4,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42699999999999999,"15min":0.42499999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.625,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.374},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.47799999999999998,"15min":0.48199999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.59499999999999997,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.52700000000000002}]},{"osd":5,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48499999999999999,"15min":0.48299999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69099999999999995,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.441}]},{"osd":6,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.373,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68200000000000005,"5min":0.754,"15min":0.754},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66000000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.40699999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":1003520,"data_stored":427068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.503,"15min":0.504},"min":{"1min":0.33400000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72699999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69299999999999995,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.316,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66000000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.46800000000000003}]},{"osd":2,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.501,"15min":0.504},"min":{"1min":0.32000000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.76400000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.501,"15min":0.501},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.65800000000000003}]},{"osd":3,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.5,"15min":0.502},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66200000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65500000000000003,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.60199999999999998}]},{"osd":4,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42799999999999999,"15min":0.42899999999999999},"min":{"1min":0.28299999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64200000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.63500000000000001}]},{"osd":5,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.67900000000000005,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.498,"15min":0.498},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.71499999999999997}]},{"osd":7,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70099999999999996,"5min":0.747,"15min":0.747},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52300000000000002,"15min":0.52200000000000002},"min":{"1min":0.38,"5min":0.37,"15min":0.37},"max":{"1min":0.752,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.82299999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1003520,"data_stored":427068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.65200000000000002}]},{"osd":1,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.433},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.54300000000000004}]},{"osd":2,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.85699999999999998}]},{"osd":3,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.90600000000000003}]},{"osd":4,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.5},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.72099999999999997}]},{"osd":6,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.69099999999999995}]},{"osd":7,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.83999999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1015808,"data_stored":432518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.58399999999999996}]},{"osd":3,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.504}]},{"osd":5,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.49099999999999999}]},{"osd":6,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.627},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.71499999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":1003520,"data_stored":427068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.37,"15min":0.35599999999999998},"min":{"1min":0.247,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.379,"15min":0.35499999999999998},"min":{"1min":0.255,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.35399999999999998}]},{"osd":1,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.38700000000000001,"15min":0.371},"min":{"1min":0.316,"5min":0.223,"15min":0.223},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.41999999999999998,"15min":0.40799999999999997},"min":{"1min":0.30099999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.311}]},{"osd":2,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.42099999999999999,"15min":0.40100000000000002},"min":{"1min":0.31,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.42199999999999999,"15min":0.40400000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.33400000000000002}]},{"osd":4,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60699999999999998,"15min":0.59499999999999997},"min":{"1min":0.48199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1160000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.59799999999999998,"15min":0.58699999999999997},"min":{"1min":0.44900000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93799999999999994,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.68600000000000005}]},{"osd":5,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.41999999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.044,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.46200000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.497}]},{"osd":6,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.61499999999999999,"15min":0.60099999999999998},"min":{"1min":0.43099999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.59999999999999998,"15min":0.58199999999999996},"min":{"1min":0.48599999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1459999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.51500000000000001}]},{"osd":7,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.51100000000000001,"5min":0.37,"15min":0.37},"max":{"1min":1.171,"5min":1.272,"15min":1.272},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.63500000000000001,"15min":0.624},"min":{"1min":0.47499999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.1020000000000001,"5min":1.302,"15min":1.302},"last":0.71599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":1003520,"data_stored":427068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.36399999999999999,"15min":0.35799999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.34699999999999998,"15min":0.33600000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.255}]},{"osd":1,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.33900000000000002,"15min":0.33300000000000002},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.40100000000000002,"15min":0.39700000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.317}]},{"osd":3,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.39900000000000002,"15min":0.39300000000000002},"min":{"1min":0.27500000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.432,"15min":0.42699999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.378}]},{"osd":4,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53500000000000003,"15min":0.52700000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85199999999999998,"5min":1.304,"15min":1.304},"last":0.56299999999999994}]},{"osd":5,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.73099999999999998,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.78500000000000003}]},{"osd":6,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.438,"5min":0.433,"15min":0.433},"max":{"1min":0.88900000000000001,"5min":1.339,"15min":1.339},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.58599999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77800000000000002,"5min":0.872,"15min":0.872},"last":0.58899999999999997}]},{"osd":7,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.44700000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.2829999999999999,"5min":1.321,"15min":1.321},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81599999999999995,"5min":1.458,"15min":1.458},"last":0.76300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":1060864,"data_stored":1017436,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":3.0569999999999999,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.221},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":2.9660000000000002,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":0.24199999999999999}]},{"osd":2,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.0289999999999999,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.28899999999999998}]},{"osd":3,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.0859999999999999,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.9950000000000001,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.5}]},{"osd":5,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.62},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.57799999999999996}]},{"osd":6,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54100000000000004}]},{"osd":7,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.64100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":1015808,"data_stored":431601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.36099999999999999,"15min":0.35599999999999998},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.38200000000000001,"15min":0.375},"min":{"1min":0.24199999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":1.016}]},{"osd":2,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.39700000000000002,"15min":0.39100000000000001},"min":{"1min":0.23400000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":1.141},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.40500000000000003,"15min":0.39800000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":1.3049999999999999}]},{"osd":3,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.36199999999999999,"15min":0.36099999999999999},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.751},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.43099999999999999,"15min":0.42399999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65900000000000003,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.995}]},{"osd":4,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.79300000000000004,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73799999999999999,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.67700000000000005}]},{"osd":5,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.40300000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79200000000000004,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.42799999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86099999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.51300000000000001}]},{"osd":6,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.879,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81399999999999995,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.92900000000000005}]},{"osd":7,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.622,"15min":0.625},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91100000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84499999999999997,"5min":1.079,"15min":1.079},"last":0.57299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:11:23.933 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T20:11:24.223 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:11:24.224 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-04-25T20:11:24.236 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-04-25T20:11:23.621971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2957,"num_read_kb":2797,"num_write":579,"num_write_kb":2844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1096,"ondisk_log_size":1096,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248756,"kb_used_data":7992,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754725964,"statfs":{"total":773094113280,"available":772839387136,"internally_reserved":0,"allocated":8183808,"data_stored":4612732,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12741,"internal_metadata":246402619},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":539,"num_read_kb":490,"num_write":41,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.190589"},"pg_stats":[{"pgid":"1.7","version":"64'117","reported_seq":324,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T20:11:16.801654+0000","last_change":"2024-04-25T20:11:03.266483+0000","last_active":"2024-04-25T20:11:16.801654+0000","last_peered":"2024-04-25T20:11:16.801654+0000","last_clean":"2024-04-25T20:11:16.801654+0000","last_became_active":"2024-04-25T20:11:03.266120+0000","last_became_peered":"2024-04-25T20:11:03.266120+0000","last_unstale":"2024-04-25T20:11:16.801654+0000","last_undegraded":"2024-04-25T20:11:16.801654+0000","last_fullsized":"2024-04-25T20:11:16.801654+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:36.703817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033361900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":159,"num_read_kb":209,"num_write":53,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"64'156","reported_seq":502,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T20:11:16.801650+0000","last_change":"2024-04-25T20:10:53.077262+0000","last_active":"2024-04-25T20:11:16.801650+0000","last_peered":"2024-04-25T20:11:16.801650+0000","last_clean":"2024-04-25T20:11:16.801650+0000","last_became_active":"2024-04-25T20:09:16.140616+0000","last_became_peered":"2024-04-25T20:09:16.140616+0000","last_unstale":"2024-04-25T20:11:16.801650+0000","last_undegraded":"2024-04-25T20:11:16.801650+0000","last_fullsized":"2024-04-25T20:11:16.801650+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:31.562175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000351966,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":530,"num_read_kb":465,"num_write":90,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"64'104","reported_seq":422,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T20:11:17.446087+0000","last_change":"2024-04-25T20:11:08.336815+0000","last_active":"2024-04-25T20:11:17.446087+0000","last_peered":"2024-04-25T20:11:17.446087+0000","last_clean":"2024-04-25T20:11:17.446087+0000","last_became_active":"2024-04-25T20:11:08.336485+0000","last_became_peered":"2024-04-25T20:11:08.336485+0000","last_unstale":"2024-04-25T20:11:17.446087+0000","last_undegraded":"2024-04-25T20:11:17.446087+0000","last_fullsized":"2024-04-25T20:11:17.446087+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:37.372937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023598099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":408,"num_read_kb":340,"num_write":32,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"64'113","reported_seq":239,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T20:11:18.038265+0000","last_change":"2024-04-25T20:11:03.270954+0000","last_active":"2024-04-25T20:11:18.038265+0000","last_peered":"2024-04-25T20:11:18.038265+0000","last_clean":"2024-04-25T20:11:18.038265+0000","last_became_active":"2024-04-25T20:11:03.270477+0000","last_became_peered":"2024-04-25T20:11:03.270477+0000","last_unstale":"2024-04-25T20:11:18.038265+0000","last_undegraded":"2024-04-25T20:11:18.038265+0000","last_fullsized":"2024-04-25T20:11:18.038265+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:53.502563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024956199999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":99,"num_read_kb":219,"num_write":57,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":226,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T20:11:18.458187+0000","last_change":"2024-04-25T20:09:17.121148+0000","last_active":"2024-04-25T20:11:18.458187+0000","last_peered":"2024-04-25T20:11:18.458187+0000","last_clean":"2024-04-25T20:11:18.458187+0000","last_became_active":"2024-04-25T20:09:17.120312+0000","last_became_peered":"2024-04-25T20:09:17.120312+0000","last_unstale":"2024-04-25T20:11:18.458187+0000","last_undegraded":"2024-04-25T20:11:18.458187+0000","last_fullsized":"2024-04-25T20:11:18.458187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_clean_scrub_stamp":"2024-04-25T20:09:15.745353+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:48.383841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"64'119","reported_seq":325,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T20:11:16.430739+0000","last_change":"2024-04-25T20:10:53.077437+0000","last_active":"2024-04-25T20:11:16.430739+0000","last_peered":"2024-04-25T20:11:16.430739+0000","last_clean":"2024-04-25T20:11:16.430739+0000","last_became_active":"2024-04-25T20:09:15.772551+0000","last_became_peered":"2024-04-25T20:09:15.772551+0000","last_unstale":"2024-04-25T20:11:16.430739+0000","last_undegraded":"2024-04-25T20:11:16.430739+0000","last_fullsized":"2024-04-25T20:11:16.430739+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:55.101025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035253999999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":240,"num_read_kb":216,"num_write":49,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"64'117","reported_seq":436,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T20:11:18.462309+0000","last_change":"2024-04-25T20:11:18.462309+0000","last_active":"2024-04-25T20:11:18.462309+0000","last_peered":"2024-04-25T20:11:18.462309+0000","last_clean":"2024-04-25T20:11:18.462309+0000","last_became_active":"2024-04-25T20:11:18.461739+0000","last_became_peered":"2024-04-25T20:11:18.461739+0000","last_unstale":"2024-04-25T20:11:18.462309+0000","last_undegraded":"2024-04-25T20:11:18.462309+0000","last_fullsized":"2024-04-25T20:11:18.462309+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:48:03.001662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001291134,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":539,"num_read_kb":490,"num_write":41,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"64'77","reported_seq":172,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T20:11:17.442232+0000","last_change":"2024-04-25T20:11:12.399325+0000","last_active":"2024-04-25T20:11:17.442232+0000","last_peered":"2024-04-25T20:11:17.442232+0000","last_clean":"2024-04-25T20:11:17.442232+0000","last_became_active":"2024-04-25T20:11:12.398945+0000","last_became_peered":"2024-04-25T20:11:12.398945+0000","last_unstale":"2024-04-25T20:11:17.442232+0000","last_undegraded":"2024-04-25T20:11:17.442232+0000","last_fullsized":"2024-04-25T20:11:17.442232+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:54:02.826800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030896500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":77,"num_write":23,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"64'176","reported_seq":655,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T20:11:18.458187+0000","last_change":"2024-04-25T20:10:53.035823+0000","last_active":"2024-04-25T20:11:18.458187+0000","last_peered":"2024-04-25T20:11:18.458187+0000","last_clean":"2024-04-25T20:11:18.458187+0000","last_became_active":"2024-04-25T20:09:16.140731+0000","last_became_peered":"2024-04-25T20:09:16.140731+0000","last_unstale":"2024-04-25T20:11:18.458187+0000","last_undegraded":"2024-04-25T20:11:18.458187+0000","last_fullsized":"2024-04-25T20:11:18.458187+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:18.458296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000234248,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":777,"num_read_kb":681,"num_write":75,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2841,"num_read_kb":2697,"num_write":420,"num_write_kb":428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":979,"ondisk_log_size":979,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1077248,"data_stored":1022905,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48499999999999999,"15min":0.48199999999999998},"min":{"1min":0.32000000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46600000000000003,"15min":0.46700000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.61499999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.46700000000000003,"15min":0.46500000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.77200000000000002}]},{"osd":2,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45700000000000002,"15min":0.45600000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.624,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.624},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.48699999999999999,"15min":0.48299999999999998},"min":{"1min":0.40200000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.66700000000000004}]},{"osd":3,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.75},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.68200000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.68200000000000005}]},{"osd":4,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42699999999999999,"15min":0.42499999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.625,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.374},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.47799999999999998,"15min":0.48199999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.59499999999999997,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.52700000000000002}]},{"osd":5,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48499999999999999,"15min":0.48299999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69099999999999995,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.441}]},{"osd":6,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.373,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68200000000000005,"5min":0.754,"15min":0.754},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66000000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.40699999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":1003520,"data_stored":427068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.503,"15min":0.504},"min":{"1min":0.33400000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72699999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69299999999999995,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.316,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66000000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.46800000000000003}]},{"osd":2,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.501,"15min":0.504},"min":{"1min":0.32000000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.76400000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.501,"15min":0.501},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.65800000000000003}]},{"osd":3,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.5,"15min":0.502},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66200000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65500000000000003,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.60199999999999998}]},{"osd":4,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42799999999999999,"15min":0.42899999999999999},"min":{"1min":0.28299999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64200000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.63500000000000001}]},{"osd":5,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.67900000000000005,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.498,"15min":0.498},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.71499999999999997}]},{"osd":7,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70099999999999996,"5min":0.747,"15min":0.747},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52300000000000002,"15min":0.52200000000000002},"min":{"1min":0.38,"5min":0.37,"15min":0.37},"max":{"1min":0.752,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.82299999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1003520,"data_stored":427068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.65200000000000002}]},{"osd":1,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.433},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.54300000000000004}]},{"osd":2,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.85699999999999998}]},{"osd":3,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.90600000000000003}]},{"osd":4,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.5},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.72099999999999997}]},{"osd":6,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.69099999999999995}]},{"osd":7,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.83999999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1015808,"data_stored":432518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.58399999999999996}]},{"osd":3,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.504}]},{"osd":5,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.49099999999999999}]},{"osd":6,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.627},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.71499999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":1003520,"data_stored":427068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.37,"15min":0.35599999999999998},"min":{"1min":0.247,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.379,"15min":0.35499999999999998},"min":{"1min":0.255,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.35399999999999998}]},{"osd":1,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.38700000000000001,"15min":0.371},"min":{"1min":0.316,"5min":0.223,"15min":0.223},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.41999999999999998,"15min":0.40799999999999997},"min":{"1min":0.30099999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.311}]},{"osd":2,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.42099999999999999,"15min":0.40100000000000002},"min":{"1min":0.31,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.42199999999999999,"15min":0.40400000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.33400000000000002}]},{"osd":4,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60699999999999998,"15min":0.59499999999999997},"min":{"1min":0.48199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1160000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.59799999999999998,"15min":0.58699999999999997},"min":{"1min":0.44900000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93799999999999994,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.68600000000000005}]},{"osd":5,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.41999999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.044,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.46200000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.497}]},{"osd":6,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.61499999999999999,"15min":0.60099999999999998},"min":{"1min":0.43099999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.59999999999999998,"15min":0.58199999999999996},"min":{"1min":0.48599999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1459999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.51500000000000001}]},{"osd":7,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.51100000000000001,"5min":0.37,"15min":0.37},"max":{"1min":1.171,"5min":1.272,"15min":1.272},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.63500000000000001,"15min":0.624},"min":{"1min":0.47499999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.1020000000000001,"5min":1.302,"15min":1.302},"last":0.71599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":1003520,"data_stored":427068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.36399999999999999,"15min":0.35799999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.34699999999999998,"15min":0.33600000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.255}]},{"osd":1,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.33900000000000002,"15min":0.33300000000000002},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.40100000000000002,"15min":0.39700000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.317}]},{"osd":3,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.39900000000000002,"15min":0.39300000000000002},"min":{"1min":0.27500000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.432,"15min":0.42699999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.378}]},{"osd":4,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53500000000000003,"15min":0.52700000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85199999999999998,"5min":1.304,"15min":1.304},"last":0.56299999999999994}]},{"osd":5,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.73099999999999998,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.78500000000000003}]},{"osd":6,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.438,"5min":0.433,"15min":0.433},"max":{"1min":0.88900000000000001,"5min":1.339,"15min":1.339},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.58599999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77800000000000002,"5min":0.872,"15min":0.872},"last":0.58899999999999997}]},{"osd":7,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.44700000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.2829999999999999,"5min":1.321,"15min":1.321},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81599999999999995,"5min":1.458,"15min":1.458},"last":0.76300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":1060864,"data_stored":1017436,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":3.0569999999999999,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.221},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":2.9660000000000002,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":0.24199999999999999}]},{"osd":2,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.0289999999999999,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.28899999999999998}]},{"osd":3,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.0859999999999999,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.9950000000000001,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.5}]},{"osd":5,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.62},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.57799999999999996}]},{"osd":6,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54100000000000004}]},{"osd":7,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.64100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":1015808,"data_stored":431601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.36099999999999999,"15min":0.35599999999999998},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.38200000000000001,"15min":0.375},"min":{"1min":0.24199999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":1.016}]},{"osd":2,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.39700000000000002,"15min":0.39100000000000001},"min":{"1min":0.23400000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":1.141},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.40500000000000003,"15min":0.39800000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":1.3049999999999999}]},{"osd":3,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.36199999999999999,"15min":0.36099999999999999},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.751},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.43099999999999999,"15min":0.42399999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65900000000000003,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.995}]},{"osd":4,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.79300000000000004,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73799999999999999,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.67700000000000005}]},{"osd":5,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.40300000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79200000000000004,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.42799999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86099999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.51300000000000001}]},{"osd":6,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.879,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81399999999999995,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.92900000000000005}]},{"osd":7,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.622,"15min":0.625},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91100000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84499999999999997,"5min":1.079,"15min":1.079},"last":0.57299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:11:24.238 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T20:11:24.238 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T20:11:24.529 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:11:24.529 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-04-25T20:11:24.542 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-04-25T20:11:23.621971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2957,"num_read_kb":2797,"num_write":579,"num_write_kb":2844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1096,"ondisk_log_size":1096,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248756,"kb_used_data":7992,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754725964,"statfs":{"total":773094113280,"available":772839387136,"internally_reserved":0,"allocated":8183808,"data_stored":4612732,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12741,"internal_metadata":246402619},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":539,"num_read_kb":490,"num_write":41,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.190589"},"pg_stats":[{"pgid":"1.7","version":"64'117","reported_seq":324,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T20:11:16.801654+0000","last_change":"2024-04-25T20:11:03.266483+0000","last_active":"2024-04-25T20:11:16.801654+0000","last_peered":"2024-04-25T20:11:16.801654+0000","last_clean":"2024-04-25T20:11:16.801654+0000","last_became_active":"2024-04-25T20:11:03.266120+0000","last_became_peered":"2024-04-25T20:11:03.266120+0000","last_unstale":"2024-04-25T20:11:16.801654+0000","last_undegraded":"2024-04-25T20:11:16.801654+0000","last_fullsized":"2024-04-25T20:11:16.801654+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:36.703817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033361900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":159,"num_read_kb":209,"num_write":53,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"64'156","reported_seq":502,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T20:11:16.801650+0000","last_change":"2024-04-25T20:10:53.077262+0000","last_active":"2024-04-25T20:11:16.801650+0000","last_peered":"2024-04-25T20:11:16.801650+0000","last_clean":"2024-04-25T20:11:16.801650+0000","last_became_active":"2024-04-25T20:09:16.140616+0000","last_became_peered":"2024-04-25T20:09:16.140616+0000","last_unstale":"2024-04-25T20:11:16.801650+0000","last_undegraded":"2024-04-25T20:11:16.801650+0000","last_fullsized":"2024-04-25T20:11:16.801650+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:31.562175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000351966,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":530,"num_read_kb":465,"num_write":90,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"64'104","reported_seq":422,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T20:11:17.446087+0000","last_change":"2024-04-25T20:11:08.336815+0000","last_active":"2024-04-25T20:11:17.446087+0000","last_peered":"2024-04-25T20:11:17.446087+0000","last_clean":"2024-04-25T20:11:17.446087+0000","last_became_active":"2024-04-25T20:11:08.336485+0000","last_became_peered":"2024-04-25T20:11:08.336485+0000","last_unstale":"2024-04-25T20:11:17.446087+0000","last_undegraded":"2024-04-25T20:11:17.446087+0000","last_fullsized":"2024-04-25T20:11:17.446087+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:37.372937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023598099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":408,"num_read_kb":340,"num_write":32,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"64'113","reported_seq":239,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T20:11:18.038265+0000","last_change":"2024-04-25T20:11:03.270954+0000","last_active":"2024-04-25T20:11:18.038265+0000","last_peered":"2024-04-25T20:11:18.038265+0000","last_clean":"2024-04-25T20:11:18.038265+0000","last_became_active":"2024-04-25T20:11:03.270477+0000","last_became_peered":"2024-04-25T20:11:03.270477+0000","last_unstale":"2024-04-25T20:11:18.038265+0000","last_undegraded":"2024-04-25T20:11:18.038265+0000","last_fullsized":"2024-04-25T20:11:18.038265+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:53.502563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024956199999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":99,"num_read_kb":219,"num_write":57,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":226,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T20:11:18.458187+0000","last_change":"2024-04-25T20:09:17.121148+0000","last_active":"2024-04-25T20:11:18.458187+0000","last_peered":"2024-04-25T20:11:18.458187+0000","last_clean":"2024-04-25T20:11:18.458187+0000","last_became_active":"2024-04-25T20:09:17.120312+0000","last_became_peered":"2024-04-25T20:09:17.120312+0000","last_unstale":"2024-04-25T20:11:18.458187+0000","last_undegraded":"2024-04-25T20:11:18.458187+0000","last_fullsized":"2024-04-25T20:11:18.458187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_clean_scrub_stamp":"2024-04-25T20:09:15.745353+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:48.383841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"64'119","reported_seq":325,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-25T20:11:16.430739+0000","last_change":"2024-04-25T20:10:53.077437+0000","last_active":"2024-04-25T20:11:16.430739+0000","last_peered":"2024-04-25T20:11:16.430739+0000","last_clean":"2024-04-25T20:11:16.430739+0000","last_became_active":"2024-04-25T20:09:15.772551+0000","last_became_peered":"2024-04-25T20:09:15.772551+0000","last_unstale":"2024-04-25T20:11:16.430739+0000","last_undegraded":"2024-04-25T20:11:16.430739+0000","last_fullsized":"2024-04-25T20:11:16.430739+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:55.101025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035253999999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":240,"num_read_kb":216,"num_write":49,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"64'117","reported_seq":436,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T20:11:18.462309+0000","last_change":"2024-04-25T20:11:18.462309+0000","last_active":"2024-04-25T20:11:18.462309+0000","last_peered":"2024-04-25T20:11:18.462309+0000","last_clean":"2024-04-25T20:11:18.462309+0000","last_became_active":"2024-04-25T20:11:18.461739+0000","last_became_peered":"2024-04-25T20:11:18.461739+0000","last_unstale":"2024-04-25T20:11:18.462309+0000","last_undegraded":"2024-04-25T20:11:18.462309+0000","last_fullsized":"2024-04-25T20:11:18.462309+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:48:03.001662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001291134,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":539,"num_read_kb":490,"num_write":41,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"64'77","reported_seq":172,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T20:11:17.442232+0000","last_change":"2024-04-25T20:11:12.399325+0000","last_active":"2024-04-25T20:11:17.442232+0000","last_peered":"2024-04-25T20:11:17.442232+0000","last_clean":"2024-04-25T20:11:17.442232+0000","last_became_active":"2024-04-25T20:11:12.398945+0000","last_became_peered":"2024-04-25T20:11:12.398945+0000","last_unstale":"2024-04-25T20:11:17.442232+0000","last_undegraded":"2024-04-25T20:11:17.442232+0000","last_fullsized":"2024-04-25T20:11:17.442232+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:54:02.826800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030896500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":77,"num_write":23,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"64'176","reported_seq":655,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T20:11:18.458187+0000","last_change":"2024-04-25T20:10:53.035823+0000","last_active":"2024-04-25T20:11:18.458187+0000","last_peered":"2024-04-25T20:11:18.458187+0000","last_clean":"2024-04-25T20:11:18.458187+0000","last_became_active":"2024-04-25T20:09:16.140731+0000","last_became_peered":"2024-04-25T20:09:16.140731+0000","last_unstale":"2024-04-25T20:11:18.458187+0000","last_undegraded":"2024-04-25T20:11:18.458187+0000","last_fullsized":"2024-04-25T20:11:18.458187+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:18.458296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000234248,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":777,"num_read_kb":681,"num_write":75,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2841,"num_read_kb":2697,"num_write":420,"num_write_kb":428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":979,"ondisk_log_size":979,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1077248,"data_stored":1022905,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48499999999999999,"15min":0.48199999999999998},"min":{"1min":0.32000000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46600000000000003,"15min":0.46700000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.61499999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.46700000000000003,"15min":0.46500000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.77200000000000002}]},{"osd":2,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45700000000000002,"15min":0.45600000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.624,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.624},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.48699999999999999,"15min":0.48299999999999998},"min":{"1min":0.40200000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.66700000000000004}]},{"osd":3,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.75},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.68200000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.68200000000000005}]},{"osd":4,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42699999999999999,"15min":0.42499999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.625,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.374},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.47799999999999998,"15min":0.48199999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.59499999999999997,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.52700000000000002}]},{"osd":5,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48499999999999999,"15min":0.48299999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69099999999999995,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.441}]},{"osd":6,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.373,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68200000000000005,"5min":0.754,"15min":0.754},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66000000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.40699999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":1003520,"data_stored":427068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.503,"15min":0.504},"min":{"1min":0.33400000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72699999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69299999999999995,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.316,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66000000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.46800000000000003}]},{"osd":2,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.501,"15min":0.504},"min":{"1min":0.32000000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.76400000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.501,"15min":0.501},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.65800000000000003}]},{"osd":3,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.5,"15min":0.502},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66200000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65500000000000003,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.60199999999999998}]},{"osd":4,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42799999999999999,"15min":0.42899999999999999},"min":{"1min":0.28299999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64200000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.63500000000000001}]},{"osd":5,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.67900000000000005,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.498,"15min":0.498},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.71499999999999997}]},{"osd":7,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70099999999999996,"5min":0.747,"15min":0.747},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52300000000000002,"15min":0.52200000000000002},"min":{"1min":0.38,"5min":0.37,"15min":0.37},"max":{"1min":0.752,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.82299999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1003520,"data_stored":427068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.65200000000000002}]},{"osd":1,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.433},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.54300000000000004}]},{"osd":2,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.85699999999999998}]},{"osd":3,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.90600000000000003}]},{"osd":4,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.5},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.72099999999999997}]},{"osd":6,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.69099999999999995}]},{"osd":7,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.83999999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1015808,"data_stored":432518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.58399999999999996}]},{"osd":3,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.504}]},{"osd":5,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.49099999999999999}]},{"osd":6,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.627},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Apr 25 20:10:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.71499999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":1003520,"data_stored":427068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.37,"15min":0.35599999999999998},"min":{"1min":0.247,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.379,"15min":0.35499999999999998},"min":{"1min":0.255,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.35399999999999998}]},{"osd":1,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.38700000000000001,"15min":0.371},"min":{"1min":0.316,"5min":0.223,"15min":0.223},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.41999999999999998,"15min":0.40799999999999997},"min":{"1min":0.30099999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.311}]},{"osd":2,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.42099999999999999,"15min":0.40100000000000002},"min":{"1min":0.31,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.42199999999999999,"15min":0.40400000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.33400000000000002}]},{"osd":4,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60699999999999998,"15min":0.59499999999999997},"min":{"1min":0.48199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1160000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.59799999999999998,"15min":0.58699999999999997},"min":{"1min":0.44900000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93799999999999994,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.68600000000000005}]},{"osd":5,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.41999999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.044,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.46200000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.497}]},{"osd":6,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.61499999999999999,"15min":0.60099999999999998},"min":{"1min":0.43099999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.59999999999999998,"15min":0.58199999999999996},"min":{"1min":0.48599999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1459999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.51500000000000001}]},{"osd":7,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.51100000000000001,"5min":0.37,"15min":0.37},"max":{"1min":1.171,"5min":1.272,"15min":1.272},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.63500000000000001,"15min":0.624},"min":{"1min":0.47499999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.1020000000000001,"5min":1.302,"15min":1.302},"last":0.71599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":1003520,"data_stored":427068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.36399999999999999,"15min":0.35799999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.34699999999999998,"15min":0.33600000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.255}]},{"osd":1,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.33900000000000002,"15min":0.33300000000000002},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.40100000000000002,"15min":0.39700000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.317}]},{"osd":3,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.39900000000000002,"15min":0.39300000000000002},"min":{"1min":0.27500000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.432,"15min":0.42699999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.378}]},{"osd":4,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53500000000000003,"15min":0.52700000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85199999999999998,"5min":1.304,"15min":1.304},"last":0.56299999999999994}]},{"osd":5,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.73099999999999998,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.78500000000000003}]},{"osd":6,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.438,"5min":0.433,"15min":0.433},"max":{"1min":0.88900000000000001,"5min":1.339,"15min":1.339},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.58599999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77800000000000002,"5min":0.872,"15min":0.872},"last":0.58899999999999997}]},{"osd":7,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.44700000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.2829999999999999,"5min":1.321,"15min":1.321},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81599999999999995,"5min":1.458,"15min":1.458},"last":0.76300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":1060864,"data_stored":1017436,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":3.0569999999999999,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.221},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":2.9660000000000002,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":0.24199999999999999}]},{"osd":2,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.0289999999999999,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.28899999999999998}]},{"osd":3,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.0859999999999999,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.9950000000000001,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.5}]},{"osd":5,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.62},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.57799999999999996}]},{"osd":6,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54100000000000004}]},{"osd":7,"last update":"Thu Apr 25 20:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.64100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":1015808,"data_stored":431601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.36099999999999999,"15min":0.35599999999999998},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.38200000000000001,"15min":0.375},"min":{"1min":0.24199999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":1.016}]},{"osd":2,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.39700000000000002,"15min":0.39100000000000001},"min":{"1min":0.23400000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":1.141},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.40500000000000003,"15min":0.39800000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":1.3049999999999999}]},{"osd":3,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.36199999999999999,"15min":0.36099999999999999},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.751},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.43099999999999999,"15min":0.42399999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65900000000000003,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.995}]},{"osd":4,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.79300000000000004,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73799999999999999,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.67700000000000005}]},{"osd":5,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.40300000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79200000000000004,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.42799999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86099999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.51300000000000001}]},{"osd":6,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.879,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81399999999999995,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.92900000000000005}]},{"osd":7,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.622,"15min":0.625},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91100000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84499999999999997,"5min":1.079,"15min":1.079},"last":0.57299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:11:24.543 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T20:11:24.839 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:11:24.839 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":86,"fsid":"975ceafd-555d-4f71-81e4-1f70364da00f","created":"2024-04-25T20:09:03.377320+0000","modified":"2024-04-25T20:11:18.439907+0000","last_up_change":"2024-04-25T20:09:13.740341+0000","last_in_change":"2024-04-25T20:10:59.204972+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-25T20:09:14.202057+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":7,"snap_epoch":63,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-04-25T20:09:15.645688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7a7118ba-4aee-404f-97e3-94c50e2996c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":31832},{"type":"v1","addr":"172.21.15.39:6818","nonce":31832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":31832},{"type":"v1","addr":"172.21.15.39:6822","nonce":31832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":31832},{"type":"v1","addr":"172.21.15.39:6830","nonce":31832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":31832},{"type":"v1","addr":"172.21.15.39:6826","nonce":31832}]},"public_addr":"172.21.15.39:6818/31832","cluster_addr":"172.21.15.39:6822/31832","heartbeat_back_addr":"172.21.15.39:6830/31832","heartbeat_front_addr":"172.21.15.39:6826/31832","state":["exists","up"]},{"osd":1,"uuid":"13637dbe-e37f-457f-b68b-8eed1b970617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":31831},{"type":"v1","addr":"172.21.15.39:6809","nonce":31831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":31831},{"type":"v1","addr":"172.21.15.39:6811","nonce":31831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":31831},{"type":"v1","addr":"172.21.15.39:6815","nonce":31831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":31831},{"type":"v1","addr":"172.21.15.39:6813","nonce":31831}]},"public_addr":"172.21.15.39:6809/31831","cluster_addr":"172.21.15.39:6811/31831","heartbeat_back_addr":"172.21.15.39:6815/31831","heartbeat_front_addr":"172.21.15.39:6813/31831","state":["exists","up"]},{"osd":2,"uuid":"29e20ce2-4776-455d-8cf4-d539e738a6a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":31829},{"type":"v1","addr":"172.21.15.39:6801","nonce":31829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":31829},{"type":"v1","addr":"172.21.15.39:6803","nonce":31829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":31829},{"type":"v1","addr":"172.21.15.39:6807","nonce":31829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":31829},{"type":"v1","addr":"172.21.15.39:6805","nonce":31829}]},"public_addr":"172.21.15.39:6801/31829","cluster_addr":"172.21.15.39:6803/31829","heartbeat_back_addr":"172.21.15.39:6807/31829","heartbeat_front_addr":"172.21.15.39:6805/31829","state":["exists","up"]},{"osd":3,"uuid":"a8b34e02-0898-4303-9fd5-62cbba9fec5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6817","nonce":31830},{"type":"v1","addr":"172.21.15.39:6819","nonce":31830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6821","nonce":31830},{"type":"v1","addr":"172.21.15.39:6823","nonce":31830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6829","nonce":31830},{"type":"v1","addr":"172.21.15.39:6831","nonce":31830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6825","nonce":31830},{"type":"v1","addr":"172.21.15.39:6827","nonce":31830}]},"public_addr":"172.21.15.39:6819/31830","cluster_addr":"172.21.15.39:6823/31830","heartbeat_back_addr":"172.21.15.39:6831/31830","heartbeat_front_addr":"172.21.15.39:6827/31830","state":["exists","up"]},{"osd":4,"uuid":"e92b5c67-6908-4fd4-9940-5cb98b5ee847","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":31724},{"type":"v1","addr":"172.21.15.135:6801","nonce":31724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":31724},{"type":"v1","addr":"172.21.15.135:6803","nonce":31724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":31724},{"type":"v1","addr":"172.21.15.135:6807","nonce":31724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":31724},{"type":"v1","addr":"172.21.15.135:6805","nonce":31724}]},"public_addr":"172.21.15.135:6801/31724","cluster_addr":"172.21.15.135:6803/31724","heartbeat_back_addr":"172.21.15.135:6807/31724","heartbeat_front_addr":"172.21.15.135:6805/31724","state":["exists","up"]},{"osd":5,"uuid":"d74d8fef-0211-413b-bff2-f97776fc1faa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":31734},{"type":"v1","addr":"172.21.15.135:6822","nonce":31734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":31734},{"type":"v1","addr":"172.21.15.135:6826","nonce":31734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":31734},{"type":"v1","addr":"172.21.15.135:6831","nonce":31734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":31734},{"type":"v1","addr":"172.21.15.135:6829","nonce":31734}]},"public_addr":"172.21.15.135:6822/31734","cluster_addr":"172.21.15.135:6826/31734","heartbeat_back_addr":"172.21.15.135:6831/31734","heartbeat_front_addr":"172.21.15.135:6829/31734","state":["exists","up"]},{"osd":6,"uuid":"7dc84c6e-76d8-4f22-8b17-51f62765762d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":31725},{"type":"v1","addr":"172.21.15.135:6809","nonce":31725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":31725},{"type":"v1","addr":"172.21.15.135:6811","nonce":31725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":31725},{"type":"v1","addr":"172.21.15.135:6815","nonce":31725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":31725},{"type":"v1","addr":"172.21.15.135:6813","nonce":31725}]},"public_addr":"172.21.15.135:6809/31725","cluster_addr":"172.21.15.135:6811/31725","heartbeat_back_addr":"172.21.15.135:6815/31725","heartbeat_front_addr":"172.21.15.135:6813/31725","state":["exists","up"]},{"osd":7,"uuid":"7adb3187-aed3-48af-868c-7230e9bfc85e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":31737},{"type":"v1","addr":"172.21.15.135:6817","nonce":31737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":31737},{"type":"v1","addr":"172.21.15.135:6819","nonce":31737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":31737},{"type":"v1","addr":"172.21.15.135:6827","nonce":31737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":31737},{"type":"v1","addr":"172.21.15.135:6823","nonce":31737}]},"public_addr":"172.21.15.135:6817/31737","cluster_addr":"172.21.15.135:6819/31737","heartbeat_back_addr":"172.21.15.135:6827/31737","heartbeat_front_addr":"172.21.15.135:6823/31737","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.683465+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.722423+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.712691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.673892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.768469+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.783897+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.686833+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:09:10.739675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T20:11:25.853 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T20:11:25.853 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T20:11:25.991 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:11:25.991 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T20:11:25.991 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:11:26.002 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T20:11:26.292 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T20:11:26.305 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T20:11:26.305 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T20:11:26.430 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:11:26.430 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T20:11:26.430 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:11:26.441 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T20:11:26.733 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T20:11:26.746 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T20:11:26.746 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T20:11:26.871 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:11:26.871 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T20:11:26.871 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:11:26.882 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T20:11:27.175 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T20:11:27.188 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T20:11:27.188 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T20:11:27.312 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:11:27.313 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T20:11:27.313 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:11:27.323 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T20:11:27.612 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T20:11:27.626 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T20:11:27.626 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T20:11:27.751 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:11:27.751 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T20:11:27.751 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:11:27.762 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T20:11:28.052 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T20:11:28.065 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T20:11:28.065 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T20:11:28.191 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:11:28.191 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T20:11:28.191 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:11:28.202 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T20:11:28.496 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T20:11:28.509 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T20:11:28.510 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T20:11:28.635 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:11:28.635 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T20:11:28.635 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:11:28.646 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T20:11:28.939 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T20:11:28.952 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T20:11:28.952 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T20:11:29.077 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-04-25T20:11:29.077 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T20:11:29.078 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-04-25T20:11:29.088 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T20:11:29.379 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T20:11:29.392 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T20:11:29.683 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:11:29.684 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-04-25T20:11:29.695 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-04-25T20:11:29.622958+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2957,"num_read_kb":2797,"num_write":579,"num_write_kb":2844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1096,"ondisk_log_size":1096,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248788,"kb_used_data":8052,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754725932,"statfs":{"total":773094113280,"available":772839354368,"internally_reserved":0,"allocated":8245248,"data_stored":4639982,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12741,"internal_metadata":246402619},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":539,"num_read_kb":490,"num_write":41,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.163291"},"pg_stats":[{"pgid":"1.7","version":"64'117","reported_seq":326,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T20:11:19.676989+0000","last_change":"2024-04-25T20:11:03.266483+0000","last_active":"2024-04-25T20:11:19.676989+0000","last_peered":"2024-04-25T20:11:19.676989+0000","last_clean":"2024-04-25T20:11:19.676989+0000","last_became_active":"2024-04-25T20:11:03.266120+0000","last_became_peered":"2024-04-25T20:11:03.266120+0000","last_unstale":"2024-04-25T20:11:19.676989+0000","last_undegraded":"2024-04-25T20:11:19.676989+0000","last_fullsized":"2024-04-25T20:11:19.676989+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:36.703817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033361900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":159,"num_read_kb":209,"num_write":53,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"64'156","reported_seq":504,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T20:11:19.676972+0000","last_change":"2024-04-25T20:10:53.077262+0000","last_active":"2024-04-25T20:11:19.676972+0000","last_peered":"2024-04-25T20:11:19.676972+0000","last_clean":"2024-04-25T20:11:19.676972+0000","last_became_active":"2024-04-25T20:09:16.140616+0000","last_became_peered":"2024-04-25T20:09:16.140616+0000","last_unstale":"2024-04-25T20:11:19.676972+0000","last_undegraded":"2024-04-25T20:11:19.676972+0000","last_fullsized":"2024-04-25T20:11:19.676972+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:31.562175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000351966,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":530,"num_read_kb":465,"num_write":90,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"64'104","reported_seq":423,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T20:11:18.456667+0000","last_change":"2024-04-25T20:11:08.336815+0000","last_active":"2024-04-25T20:11:18.456667+0000","last_peered":"2024-04-25T20:11:18.456667+0000","last_clean":"2024-04-25T20:11:18.456667+0000","last_became_active":"2024-04-25T20:11:08.336485+0000","last_became_peered":"2024-04-25T20:11:08.336485+0000","last_unstale":"2024-04-25T20:11:18.456667+0000","last_undegraded":"2024-04-25T20:11:18.456667+0000","last_fullsized":"2024-04-25T20:11:18.456667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:37.372937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023598099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":408,"num_read_kb":340,"num_write":32,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"64'113","reported_seq":239,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T20:11:18.038265+0000","last_change":"2024-04-25T20:11:03.270954+0000","last_active":"2024-04-25T20:11:18.038265+0000","last_peered":"2024-04-25T20:11:18.038265+0000","last_clean":"2024-04-25T20:11:18.038265+0000","last_became_active":"2024-04-25T20:11:03.270477+0000","last_became_peered":"2024-04-25T20:11:03.270477+0000","last_unstale":"2024-04-25T20:11:18.038265+0000","last_undegraded":"2024-04-25T20:11:18.038265+0000","last_fullsized":"2024-04-25T20:11:18.038265+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:53.502563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024956199999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":99,"num_read_kb":219,"num_write":57,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":226,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T20:11:18.458187+0000","last_change":"2024-04-25T20:09:17.121148+0000","last_active":"2024-04-25T20:11:18.458187+0000","last_peered":"2024-04-25T20:11:18.458187+0000","last_clean":"2024-04-25T20:11:18.458187+0000","last_became_active":"2024-04-25T20:09:17.120312+0000","last_became_peered":"2024-04-25T20:09:17.120312+0000","last_unstale":"2024-04-25T20:11:18.458187+0000","last_undegraded":"2024-04-25T20:11:18.458187+0000","last_fullsized":"2024-04-25T20:11:18.458187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:15.745353+0000","last_clean_scrub_stamp":"2024-04-25T20:09:15.745353+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:48.383841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"64'119","reported_seq":327,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T20:11:20.029780+0000","last_change":"2024-04-25T20:10:53.077437+0000","last_active":"2024-04-25T20:11:20.029780+0000","last_peered":"2024-04-25T20:11:20.029780+0000","last_clean":"2024-04-25T20:11:20.029780+0000","last_became_active":"2024-04-25T20:09:15.772551+0000","last_became_peered":"2024-04-25T20:09:15.772551+0000","last_unstale":"2024-04-25T20:11:20.029780+0000","last_undegraded":"2024-04-25T20:11:20.029780+0000","last_fullsized":"2024-04-25T20:11:20.029780+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:55.101025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035253999999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":240,"num_read_kb":216,"num_write":49,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"64'117","reported_seq":436,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T20:11:18.462309+0000","last_change":"2024-04-25T20:11:18.462309+0000","last_active":"2024-04-25T20:11:18.462309+0000","last_peered":"2024-04-25T20:11:18.462309+0000","last_clean":"2024-04-25T20:11:18.462309+0000","last_became_active":"2024-04-25T20:11:18.461739+0000","last_became_peered":"2024-04-25T20:11:18.461739+0000","last_unstale":"2024-04-25T20:11:18.462309+0000","last_undegraded":"2024-04-25T20:11:18.462309+0000","last_fullsized":"2024-04-25T20:11:18.462309+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:48:03.001662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001291134,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":539,"num_read_kb":490,"num_write":41,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"64'77","reported_seq":173,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T20:11:19.645232+0000","last_change":"2024-04-25T20:11:12.399325+0000","last_active":"2024-04-25T20:11:19.645232+0000","last_peered":"2024-04-25T20:11:19.645232+0000","last_clean":"2024-04-25T20:11:19.645232+0000","last_became_active":"2024-04-25T20:11:12.398945+0000","last_became_peered":"2024-04-25T20:11:12.398945+0000","last_unstale":"2024-04-25T20:11:19.645232+0000","last_undegraded":"2024-04-25T20:11:19.645232+0000","last_fullsized":"2024-04-25T20:11:19.645232+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:54:02.826800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030896500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":77,"num_write":23,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"64'176","reported_seq":655,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T20:11:18.458187+0000","last_change":"2024-04-25T20:10:53.035823+0000","last_active":"2024-04-25T20:11:18.458187+0000","last_peered":"2024-04-25T20:11:18.458187+0000","last_clean":"2024-04-25T20:11:18.458187+0000","last_became_active":"2024-04-25T20:09:16.140731+0000","last_became_peered":"2024-04-25T20:09:16.140731+0000","last_unstale":"2024-04-25T20:11:18.458187+0000","last_undegraded":"2024-04-25T20:11:18.458187+0000","last_fullsized":"2024-04-25T20:11:18.458187+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:09:14.743556+0000","last_clean_scrub_stamp":"2024-04-25T20:09:14.743556+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:18.458296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000234248,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":777,"num_read_kb":681,"num_write":75,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2841,"num_read_kb":2697,"num_write":420,"num_write_kb":428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":979,"ondisk_log_size":979,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1077248,"data_stored":1022905,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48499999999999999,"15min":0.48199999999999998},"min":{"1min":0.32000000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46600000000000003,"15min":0.46700000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.56299999999999994}]},{"osd":1,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.61499999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.622},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.46700000000000003,"15min":0.46500000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.54800000000000004}]},{"osd":2,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45700000000000002,"15min":0.45600000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.624,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.48699999999999999,"15min":0.48299999999999998},"min":{"1min":0.40200000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.443}]},{"osd":3,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.68200000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.52400000000000002}]},{"osd":4,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42699999999999999,"15min":0.42499999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.625,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.47799999999999998,"15min":0.48199999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.59499999999999997,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.47599999999999998}]},{"osd":5,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48499999999999999,"15min":0.48299999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69099999999999995,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.373,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68200000000000005,"5min":0.754,"15min":0.754},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66000000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.51600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":1015808,"data_stored":432518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.503,"15min":0.504},"min":{"1min":0.33400000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72699999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69299999999999995,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.316,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.434},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66000000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.45100000000000001}]},{"osd":2,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.501,"15min":0.504},"min":{"1min":0.32000000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.76400000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.501,"15min":0.501},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.60899999999999999}]},{"osd":3,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.5,"15min":0.502},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66200000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65500000000000003,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.50900000000000001}]},{"osd":4,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42799999999999999,"15min":0.42899999999999999},"min":{"1min":0.28299999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64200000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.54500000000000004}]},{"osd":5,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.67900000000000005,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.63},{"interface":"front","average":{"1min":0.5,"5min":0.498,"15min":0.498},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.64800000000000002}]},{"osd":7,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70099999999999996,"5min":0.747,"15min":0.747},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52300000000000002,"15min":0.52200000000000002},"min":{"1min":0.38,"5min":0.37,"15min":0.37},"max":{"1min":0.752,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.56399999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1015808,"data_stored":432518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.503}]},{"osd":1,"last update":"Thu Apr 25 20:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.44400000000000001,"15min":0.443},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.48999999999999999,"15min":0.49299999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68200000000000005,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.55200000000000005}]},{"osd":2,"last update":"Thu Apr 25 20:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.505,"15min":0.5},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.62}]},{"osd":3,"last update":"Thu Apr 25 20:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.501,"15min":0.499},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.60799999999999998}]},{"osd":4,"last update":"Thu Apr 25 20:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.443,"15min":0.44500000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.61199999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.435},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55200000000000005,"15min":0.55300000000000005},"min":{"1min":0.378,"5min":0.31,"15min":0.31},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.58199999999999996}]},{"osd":6,"last update":"Thu Apr 25 20:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.502,"15min":0.501},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.56399999999999995}]},{"osd":7,"last update":"Thu Apr 25 20:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.45900000000000002,"15min":0.45500000000000002},"min":{"1min":0.30599999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.51900000000000002,"15min":0.51300000000000001},"min":{"1min":0.36099999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.48299999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1015808,"data_stored":432518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.47299999999999998,"15min":0.47699999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.60399999999999998,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.38600000000000001}]},{"osd":1,"last update":"Thu Apr 25 20:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.502,"15min":0.48899999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.0910000000000002,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.495,"15min":0.497},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64800000000000002,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.40500000000000003}]},{"osd":2,"last update":"Thu Apr 25 20:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.34999999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51900000000000002,"15min":0.52500000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69099999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Apr 25 20:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.505,"15min":0.51100000000000001},"min":{"1min":0.375,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63300000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.54100000000000004,"15min":0.55200000000000005},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.73699999999999999,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.60899999999999999}]},{"osd":5,"last update":"Thu Apr 25 20:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.48899999999999999,"15min":0.505},"min":{"1min":0.27700000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.61299999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.47899999999999998,"15min":0.49099999999999999},"min":{"1min":0.29699999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.66600000000000004,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.433}]},{"osd":6,"last update":"Thu Apr 25 20:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.48699999999999999,"15min":0.499},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.65900000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.53800000000000003,"15min":0.55200000000000005},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.68400000000000005,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.54900000000000004}]},{"osd":7,"last update":"Thu Apr 25 20:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.54000000000000004,"15min":0.55600000000000005},"min":{"1min":0.32600000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.57399999999999995,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.54300000000000004,"15min":0.56200000000000006},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71499999999999997,"5min":1.042,"15min":1.042},"last":0.56000000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":1015808,"data_stored":432518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.37,"15min":0.35599999999999998},"min":{"1min":0.247,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.24299999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.379,"15min":0.35499999999999998},"min":{"1min":0.255,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.25900000000000001}]},{"osd":1,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.38700000000000001,"15min":0.371},"min":{"1min":0.316,"5min":0.223,"15min":0.223},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.41999999999999998,"15min":0.40799999999999997},"min":{"1min":0.30099999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.28000000000000003}]},{"osd":2,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.42099999999999999,"15min":0.40100000000000002},"min":{"1min":0.31,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.42199999999999999,"15min":0.40400000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.33700000000000002}]},{"osd":4,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60699999999999998,"15min":0.59499999999999997},"min":{"1min":0.48199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1160000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.2909999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.59799999999999998,"15min":0.58699999999999997},"min":{"1min":0.44900000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93799999999999994,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":1.3280000000000001}]},{"osd":5,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.41999999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.044,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.46200000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.54000000000000004}]},{"osd":6,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.61499999999999999,"15min":0.60099999999999998},"min":{"1min":0.43099999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":1.3140000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.59999999999999998,"15min":0.58199999999999996},"min":{"1min":0.48599999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1459999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.51100000000000001}]},{"osd":7,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.51100000000000001,"5min":0.37,"15min":0.37},"max":{"1min":1.171,"5min":1.272,"15min":1.272},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.63500000000000001,"15min":0.624},"min":{"1min":0.47499999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.1020000000000001,"5min":1.302,"15min":1.302},"last":0.61299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":1015808,"data_stored":432518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.36399999999999999,"15min":0.35799999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.34699999999999998,"15min":0.33600000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.33900000000000002,"15min":0.33300000000000002},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.40100000000000002,"15min":0.39700000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.36599999999999999}]},{"osd":3,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.39900000000000002,"15min":0.39300000000000002},"min":{"1min":0.27500000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.432,"15min":0.42699999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.41499999999999998}]},{"osd":4,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53500000000000003,"15min":0.52700000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85199999999999998,"5min":1.304,"15min":1.304},"last":0.502}]},{"osd":5,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.73099999999999998,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.438,"5min":0.433,"15min":0.433},"max":{"1min":0.88900000000000001,"5min":1.339,"15min":1.339},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.58599999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77800000000000002,"5min":0.872,"15min":0.872},"last":0.44600000000000001}]},{"osd":7,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.44700000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.2829999999999999,"5min":1.321,"15min":1.321},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81599999999999995,"5min":1.458,"15min":1.458},"last":0.55200000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1073152,"data_stored":1022886,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.45500000000000002,"15min":0.46600000000000003},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.68400000000000005,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.47399999999999998,"15min":0.47799999999999998},"min":{"1min":0.23699999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.4370000000000001,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":0.26900000000000002}]},{"osd":2,"last update":"Thu Apr 25 20:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.46000000000000002,"15min":0.47699999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.67100000000000004,"5min":2.843,"15min":2.843},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.497,"15min":0.50700000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70699999999999996,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.38900000000000001}]},{"osd":3,"last update":"Thu Apr 25 20:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.52300000000000002,"15min":0.53700000000000003},"min":{"1min":0.27700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.66100000000000003,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":0.441},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.54000000000000004,"15min":0.55500000000000005},"min":{"1min":0.312,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.0569999999999999,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":0.31}]},{"osd":4,"last update":"Thu Apr 25 20:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.39000000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80200000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.63300000000000001,"15min":0.63900000000000001},"min":{"1min":0.46200000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81899999999999995,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.47899999999999998}]},{"osd":5,"last update":"Thu Apr 25 20:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.42999999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56799999999999995,"15min":0.56599999999999995},"min":{"1min":0.373,"5min":0.37,"15min":0.37},"max":{"1min":0.79200000000000004,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":1.1160000000000001}]},{"osd":6,"last update":"Thu Apr 25 20:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.44700000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86099999999999999,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.61199999999999999,"15min":0.59899999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.45200000000000001}]},{"osd":7,"last update":"Thu Apr 25 20:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.65400000000000003,"15min":0.64700000000000002},"min":{"1min":0.46600000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.52,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.145},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61599999999999999,"15min":0.61899999999999999},"min":{"1min":0.41499999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.755,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.40100000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":1015808,"data_stored":431601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.36099999999999999,"15min":0.35599999999999998},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.378},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.38200000000000001,"15min":0.375},"min":{"1min":0.24199999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.48599999999999999}]},{"osd":2,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.39700000000000002,"15min":0.39100000000000001},"min":{"1min":0.23400000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.40500000000000003,"15min":0.39800000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.38800000000000001}]},{"osd":3,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.36199999999999999,"15min":0.36099999999999999},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.43099999999999999,"15min":0.42399999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65900000000000003,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.42699999999999999}]},{"osd":4,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.79300000000000004,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.877},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73799999999999999,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.85999999999999999}]},{"osd":5,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.40300000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79200000000000004,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.42799999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86099999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.82799999999999996}]},{"osd":6,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.879,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81399999999999995,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.63200000000000001}]},{"osd":7,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.622,"15min":0.625},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91100000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84499999999999997,"5min":1.079,"15min":1.079},"last":0.75900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:11:29.700 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-04-25T20:09:14.743556+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=20, tm_min=9, tm_sec=14, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=20, tm_min=11, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T20:11:29.700 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-04-25T20:09:14.743556+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=20, tm_min=9, tm_sec=14, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=20, tm_min=11, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T20:11:29.700 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-04-25T20:09:14.743556+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=20, tm_min=9, tm_sec=14, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=20, tm_min=11, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T20:11:29.700 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-04-25T20:09:14.743556+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=20, tm_min=9, tm_sec=14, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=20, tm_min=11, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T20:11:29.700 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-25T20:09:15.745353+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=20, tm_min=9, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=20, tm_min=11, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T20:11:29.701 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-04-25T20:09:14.743556+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=20, tm_min=9, tm_sec=14, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=20, tm_min=11, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T20:11:29.701 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-04-25T20:09:14.743556+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=20, tm_min=9, tm_sec=14, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=20, tm_min=11, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T20:11:29.701 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-04-25T20:09:14.743556+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=20, tm_min=9, tm_sec=14, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=20, tm_min=11, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T20:11:29.701 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T20:09:14.743556+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=20, tm_min=9, tm_sec=14, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=20, tm_min=11, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T20:11:29.701 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T20:11:49.702 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T20:11:50.006 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-04-25T20:11:50.006 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-04-25T20:11:50.018 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-04-25T20:11:49.627273+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2957,"num_read_kb":2797,"num_write":579,"num_write_kb":2844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1102,"ondisk_log_size":1102,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248760,"kb_used_data":8052,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754725960,"statfs":{"total":773094113280,"available":772839383040,"internally_reserved":0,"allocated":8245248,"data_stored":4639982,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12741,"internal_metadata":246402619},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002519"},"pg_stats":[{"pgid":"1.7","version":"64'117","reported_seq":338,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T20:11:30.763349+0000","last_change":"2024-04-25T20:11:30.763483+0000","last_active":"2024-04-25T20:11:30.763349+0000","last_peered":"2024-04-25T20:11:30.763349+0000","last_clean":"2024-04-25T20:11:30.763349+0000","last_became_active":"2024-04-25T20:11:03.266120+0000","last_became_peered":"2024-04-25T20:11:03.266120+0000","last_unstale":"2024-04-25T20:11:30.763349+0000","last_undegraded":"2024-04-25T20:11:30.763349+0000","last_fullsized":"2024-04-25T20:11:30.763349+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'117","last_scrub_stamp":"2024-04-25T20:11:30.763324+0000","last_deep_scrub":"64'117","last_deep_scrub_stamp":"2024-04-25T20:11:30.763324+0000","last_clean_scrub_stamp":"2024-04-25T20:11:30.763324+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:40:37.592883+0000","scrub_duration":0.0020043410000000002,"objects_trimmed":0,"snaptrim_duration":0.00033361900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":159,"num_read_kb":209,"num_write":53,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"64'156","reported_seq":516,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T20:11:31.788308+0000","last_change":"2024-04-25T20:11:31.788439+0000","last_active":"2024-04-25T20:11:31.788308+0000","last_peered":"2024-04-25T20:11:31.788308+0000","last_clean":"2024-04-25T20:11:31.788308+0000","last_became_active":"2024-04-25T20:09:16.140616+0000","last_became_peered":"2024-04-25T20:09:16.140616+0000","last_unstale":"2024-04-25T20:11:31.788308+0000","last_undegraded":"2024-04-25T20:11:31.788308+0000","last_fullsized":"2024-04-25T20:11:31.788308+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'156","last_scrub_stamp":"2024-04-25T20:11:31.788267+0000","last_deep_scrub":"64'156","last_deep_scrub_stamp":"2024-04-25T20:11:31.788267+0000","last_clean_scrub_stamp":"2024-04-25T20:11:31.788267+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:12.622470+0000","scrub_duration":0.0022123400000000001,"objects_trimmed":0,"snaptrim_duration":0.000351966,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":530,"num_read_kb":465,"num_write":90,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"64'104","reported_seq":435,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T20:11:28.401888+0000","last_change":"2024-04-25T20:11:28.402020+0000","last_active":"2024-04-25T20:11:28.401888+0000","last_peered":"2024-04-25T20:11:28.401888+0000","last_clean":"2024-04-25T20:11:28.401888+0000","last_became_active":"2024-04-25T20:11:08.336485+0000","last_became_peered":"2024-04-25T20:11:08.336485+0000","last_unstale":"2024-04-25T20:11:28.401888+0000","last_undegraded":"2024-04-25T20:11:28.401888+0000","last_fullsized":"2024-04-25T20:11:28.401888+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'104","last_scrub_stamp":"2024-04-25T20:11:28.401864+0000","last_deep_scrub":"64'104","last_deep_scrub_stamp":"2024-04-25T20:11:28.401864+0000","last_clean_scrub_stamp":"2024-04-25T20:11:28.401864+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:49.033417+0000","scrub_duration":0.0013686359999999999,"objects_trimmed":0,"snaptrim_duration":0.00023598099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":408,"num_read_kb":340,"num_write":32,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"86'114","reported_seq":252,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T20:11:27.949519+0000","last_change":"2024-04-25T20:11:27.949605+0000","last_active":"2024-04-25T20:11:27.949519+0000","last_peered":"2024-04-25T20:11:27.949519+0000","last_clean":"2024-04-25T20:11:27.949519+0000","last_became_active":"2024-04-25T20:11:03.270477+0000","last_became_peered":"2024-04-25T20:11:03.270477+0000","last_unstale":"2024-04-25T20:11:27.949519+0000","last_undegraded":"2024-04-25T20:11:27.949519+0000","last_fullsized":"2024-04-25T20:11:27.949519+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'114","last_scrub_stamp":"2024-04-25T20:11:27.949500+0000","last_deep_scrub":"86'114","last_deep_scrub_stamp":"2024-04-25T20:11:27.949500+0000","last_clean_scrub_stamp":"2024-04-25T20:11:27.949500+0000","objects_scrubbed":1,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:23:09.832239+0000","scrub_duration":0.003265404,"objects_trimmed":0,"snaptrim_duration":0.00024956199999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":99,"num_read_kb":219,"num_write":57,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"86'119","reported_seq":240,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T20:11:36.315668+0000","last_change":"2024-04-25T20:11:36.315790+0000","last_active":"2024-04-25T20:11:36.315668+0000","last_peered":"2024-04-25T20:11:36.315668+0000","last_clean":"2024-04-25T20:11:36.315668+0000","last_became_active":"2024-04-25T20:09:17.120312+0000","last_became_peered":"2024-04-25T20:09:17.120312+0000","last_unstale":"2024-04-25T20:11:36.315668+0000","last_undegraded":"2024-04-25T20:11:36.315668+0000","last_fullsized":"2024-04-25T20:11:36.315668+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'119","last_scrub_stamp":"2024-04-25T20:11:36.315644+0000","last_deep_scrub":"86'119","last_deep_scrub_stamp":"2024-04-25T20:11:36.315644+0000","last_clean_scrub_stamp":"2024-04-25T20:11:36.315644+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:04.969293+0000","scrub_duration":0.0069486469999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"86'120","reported_seq":340,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T20:11:37.320387+0000","last_change":"2024-04-25T20:11:37.320491+0000","last_active":"2024-04-25T20:11:37.320387+0000","last_peered":"2024-04-25T20:11:37.320387+0000","last_clean":"2024-04-25T20:11:37.320387+0000","last_became_active":"2024-04-25T20:09:15.772551+0000","last_became_peered":"2024-04-25T20:09:15.772551+0000","last_unstale":"2024-04-25T20:11:37.320387+0000","last_undegraded":"2024-04-25T20:11:37.320387+0000","last_fullsized":"2024-04-25T20:11:37.320387+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'120","last_scrub_stamp":"2024-04-25T20:11:37.320363+0000","last_deep_scrub":"86'120","last_deep_scrub_stamp":"2024-04-25T20:11:37.320363+0000","last_clean_scrub_stamp":"2024-04-25T20:11:37.320363+0000","objects_scrubbed":1,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:40:25.006220+0000","scrub_duration":0.0038414870000000002,"objects_trimmed":0,"snaptrim_duration":0.00035253999999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":240,"num_read_kb":216,"num_write":49,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"86'119","reported_seq":450,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T20:11:37.337194+0000","last_change":"2024-04-25T20:11:37.337293+0000","last_active":"2024-04-25T20:11:37.337194+0000","last_peered":"2024-04-25T20:11:37.337194+0000","last_clean":"2024-04-25T20:11:37.337194+0000","last_became_active":"2024-04-25T20:11:18.461739+0000","last_became_peered":"2024-04-25T20:11:18.461739+0000","last_unstale":"2024-04-25T20:11:37.337194+0000","last_undegraded":"2024-04-25T20:11:37.337194+0000","last_fullsized":"2024-04-25T20:11:37.337194+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'119","last_scrub_stamp":"2024-04-25T20:11:37.337169+0000","last_deep_scrub":"86'119","last_deep_scrub_stamp":"2024-04-25T20:11:37.337169+0000","last_clean_scrub_stamp":"2024-04-25T20:11:37.337169+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:03.684506+0000","scrub_duration":0.0065704800000000001,"objects_trimmed":0,"snaptrim_duration":0.001291134,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":539,"num_read_kb":490,"num_write":41,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"64'77","reported_seq":185,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T20:11:41.921430+0000","last_change":"2024-04-25T20:11:41.921558+0000","last_active":"2024-04-25T20:11:41.921430+0000","last_peered":"2024-04-25T20:11:41.921430+0000","last_clean":"2024-04-25T20:11:41.921430+0000","last_became_active":"2024-04-25T20:11:12.398945+0000","last_became_peered":"2024-04-25T20:11:12.398945+0000","last_unstale":"2024-04-25T20:11:41.921430+0000","last_undegraded":"2024-04-25T20:11:41.921430+0000","last_fullsized":"2024-04-25T20:11:41.921430+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'77","last_scrub_stamp":"2024-04-25T20:11:41.921399+0000","last_deep_scrub":"64'77","last_deep_scrub_stamp":"2024-04-25T20:11:41.921399+0000","last_clean_scrub_stamp":"2024-04-25T20:11:41.921399+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:04:48.520093+0000","scrub_duration":0.0020750690000000001,"objects_trimmed":0,"snaptrim_duration":0.00030896500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":77,"num_write":23,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"64'176","reported_seq":667,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T20:11:33.249686+0000","last_change":"2024-04-25T20:11:33.249796+0000","last_active":"2024-04-25T20:11:33.249686+0000","last_peered":"2024-04-25T20:11:33.249686+0000","last_clean":"2024-04-25T20:11:33.249686+0000","last_became_active":"2024-04-25T20:09:16.140731+0000","last_became_peered":"2024-04-25T20:09:16.140731+0000","last_unstale":"2024-04-25T20:11:33.249686+0000","last_undegraded":"2024-04-25T20:11:33.249686+0000","last_fullsized":"2024-04-25T20:11:33.249686+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'176","last_scrub_stamp":"2024-04-25T20:11:33.249663+0000","last_deep_scrub":"64'176","last_deep_scrub_stamp":"2024-04-25T20:11:33.249663+0000","last_clean_scrub_stamp":"2024-04-25T20:11:33.249663+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:47:00.610677+0000","scrub_duration":0.0020418559999999999,"objects_trimmed":0,"snaptrim_duration":0.000234248,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":777,"num_read_kb":681,"num_write":75,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":18156,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2841,"num_read_kb":2697,"num_write":420,"num_write_kb":428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":983,"ondisk_log_size":983,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1077248,"data_stored":1022905,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48499999999999999,"15min":0.48199999999999998},"min":{"1min":0.32000000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46600000000000003,"15min":0.46700000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.505}]},{"osd":1,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.61499999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.46700000000000003,"15min":0.46500000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45700000000000002,"15min":0.45600000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.624,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.48699999999999999,"15min":0.48299999999999998},"min":{"1min":0.40200000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.68200000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.63}]},{"osd":4,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42699999999999999,"15min":0.42499999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.625,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.47799999999999998,"15min":0.48199999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.59499999999999997,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.30599999999999999}]},{"osd":5,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48499999999999999,"15min":0.48299999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69099999999999995,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.41999999999999998}]},{"osd":6,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.373,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68200000000000005,"5min":0.754,"15min":0.754},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66000000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.46300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":1015808,"data_stored":432518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.503,"15min":0.504},"min":{"1min":0.33400000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72699999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69299999999999995,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.72299999999999998}]},{"osd":1,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.316,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66000000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.64900000000000002}]},{"osd":2,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.501,"15min":0.504},"min":{"1min":0.32000000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.76400000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.501,"15min":0.501},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.54500000000000004}]},{"osd":3,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.5,"15min":0.502},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66200000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.753},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65500000000000003,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.55700000000000005}]},{"osd":4,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.38},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42799999999999999,"15min":0.42899999999999999},"min":{"1min":0.28299999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64200000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.40400000000000003}]},{"osd":5,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.67900000000000005,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.498,"15min":0.498},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.46500000000000002}]},{"osd":7,"last update":"Thu Apr 25 20:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70099999999999996,"5min":0.747,"15min":0.747},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52300000000000002,"15min":0.52200000000000002},"min":{"1min":0.38,"5min":0.37,"15min":0.37},"max":{"1min":0.752,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.51300000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1015808,"data_stored":432518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.63},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.52700000000000002}]},{"osd":1,"last update":"Thu Apr 25 20:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.44400000000000001,"15min":0.443},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.48999999999999999,"15min":0.49299999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68200000000000005,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.67500000000000004}]},{"osd":2,"last update":"Thu Apr 25 20:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.505,"15min":0.5},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.68999999999999995}]},{"osd":3,"last update":"Thu Apr 25 20:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.501,"15min":0.499},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.64700000000000002}]},{"osd":4,"last update":"Thu Apr 25 20:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.443,"15min":0.44500000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.61199999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55200000000000005,"15min":0.55300000000000005},"min":{"1min":0.378,"5min":0.31,"15min":0.31},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.57199999999999995}]},{"osd":6,"last update":"Thu Apr 25 20:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.502,"15min":0.501},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.46899999999999997}]},{"osd":7,"last update":"Thu Apr 25 20:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.45900000000000002,"15min":0.45500000000000002},"min":{"1min":0.30599999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.51900000000000002,"15min":0.51300000000000001},"min":{"1min":0.36099999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.53700000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1015808,"data_stored":432518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.47299999999999998,"15min":0.47699999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.60399999999999998,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.48599999999999999}]},{"osd":1,"last update":"Thu Apr 25 20:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.502,"15min":0.48899999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.0910000000000002,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.495,"15min":0.497},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64800000000000002,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.44}]},{"osd":2,"last update":"Thu Apr 25 20:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.34999999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51900000000000002,"15min":0.52500000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69099999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.46000000000000002}]},{"osd":3,"last update":"Thu Apr 25 20:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.505,"15min":0.51100000000000001},"min":{"1min":0.375,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63300000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.54100000000000004,"15min":0.55200000000000005},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.73699999999999999,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.42799999999999999}]},{"osd":5,"last update":"Thu Apr 25 20:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.48899999999999999,"15min":0.505},"min":{"1min":0.27700000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.61299999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.5},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.47899999999999998,"15min":0.49099999999999999},"min":{"1min":0.29699999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.66600000000000004,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.45000000000000001}]},{"osd":6,"last update":"Thu Apr 25 20:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.48699999999999999,"15min":0.499},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.65900000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.53800000000000003,"15min":0.55200000000000005},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.68400000000000005,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.34499999999999997}]},{"osd":7,"last update":"Thu Apr 25 20:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.54000000000000004,"15min":0.55600000000000005},"min":{"1min":0.32600000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.57399999999999995,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.54300000000000004,"15min":0.56200000000000006},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71499999999999997,"5min":1.042,"15min":1.042},"last":0.36599999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":1015808,"data_stored":432518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.37,"15min":0.35599999999999998},"min":{"1min":0.247,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.379,"15min":0.35499999999999998},"min":{"1min":0.255,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.56399999999999995}]},{"osd":1,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.38700000000000001,"15min":0.371},"min":{"1min":0.316,"5min":0.223,"15min":0.223},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.41999999999999998,"15min":0.40799999999999997},"min":{"1min":0.30099999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.505}]},{"osd":2,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.42099999999999999,"15min":0.40100000000000002},"min":{"1min":0.31,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.42199999999999999,"15min":0.40400000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.54700000000000004}]},{"osd":4,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60699999999999998,"15min":0.59499999999999997},"min":{"1min":0.48199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1160000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.59799999999999998,"15min":0.58699999999999997},"min":{"1min":0.44900000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93799999999999994,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.76500000000000001}]},{"osd":5,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.41999999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.044,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.46200000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.71099999999999997}]},{"osd":6,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.61499999999999999,"15min":0.60099999999999998},"min":{"1min":0.43099999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.59999999999999998,"15min":0.58199999999999996},"min":{"1min":0.48599999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1459999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.746}]},{"osd":7,"last update":"Thu Apr 25 20:11:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.51100000000000001,"5min":0.37,"15min":0.37},"max":{"1min":1.171,"5min":1.272,"15min":1.272},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.63500000000000001,"15min":0.624},"min":{"1min":0.47499999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.1020000000000001,"5min":1.302,"15min":1.302},"last":0.82699999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":1015808,"data_stored":432518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.36399999999999999,"15min":0.35799999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.34699999999999998,"15min":0.33600000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.317}]},{"osd":1,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.33900000000000002,"15min":0.33300000000000002},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.40100000000000002,"15min":0.39700000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.442}]},{"osd":3,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.39900000000000002,"15min":0.39300000000000002},"min":{"1min":0.27500000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.432,"15min":0.42699999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.48599999999999999}]},{"osd":4,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53500000000000003,"15min":0.52700000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85199999999999998,"5min":1.304,"15min":1.304},"last":0.56000000000000005}]},{"osd":5,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.73099999999999998,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.621}]},{"osd":6,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.438,"5min":0.433,"15min":0.433},"max":{"1min":0.88900000000000001,"5min":1.339,"15min":1.339},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.58599999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77800000000000002,"5min":0.872,"15min":0.872},"last":0.72399999999999998}]},{"osd":7,"last update":"Thu Apr 25 20:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.44700000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.2829999999999999,"5min":1.321,"15min":1.321},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81599999999999995,"5min":1.458,"15min":1.458},"last":0.68500000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":1073152,"data_stored":1022886,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 20:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.45500000000000002,"15min":0.46600000000000003},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.68400000000000005,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.47399999999999998,"15min":0.47799999999999998},"min":{"1min":0.23699999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.4370000000000001,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":0.39400000000000002}]},{"osd":2,"last update":"Thu Apr 25 20:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.46000000000000002,"15min":0.47699999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.67100000000000004,"5min":2.843,"15min":2.843},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.44,"5min":0.497,"15min":0.50700000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70699999999999996,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.41499999999999998}]},{"osd":3,"last update":"Thu Apr 25 20:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.52300000000000002,"15min":0.53700000000000003},"min":{"1min":0.27700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.66100000000000003,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.54000000000000004,"15min":0.55500000000000005},"min":{"1min":0.312,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.0569999999999999,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":0.432}]},{"osd":4,"last update":"Thu Apr 25 20:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.39000000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80200000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.63300000000000001,"15min":0.63900000000000001},"min":{"1min":0.46200000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81899999999999995,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.64700000000000002}]},{"osd":5,"last update":"Thu Apr 25 20:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.42999999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56799999999999995,"15min":0.56599999999999995},"min":{"1min":0.373,"5min":0.37,"15min":0.37},"max":{"1min":0.79200000000000004,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.54100000000000004}]},{"osd":6,"last update":"Thu Apr 25 20:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.44700000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86099999999999999,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.61199999999999999,"15min":0.59899999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.65900000000000003}]},{"osd":7,"last update":"Thu Apr 25 20:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.65400000000000003,"15min":0.64700000000000002},"min":{"1min":0.46600000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.52,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61599999999999999,"15min":0.61899999999999999},"min":{"1min":0.41499999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.755,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.61299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":1015808,"data_stored":431601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.36099999999999999,"15min":0.35599999999999998},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.38200000000000001,"15min":0.375},"min":{"1min":0.24199999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.40500000000000003}]},{"osd":2,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.39700000000000002,"15min":0.39100000000000001},"min":{"1min":0.23400000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.40500000000000003,"15min":0.39800000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.33900000000000002}]},{"osd":3,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.36199999999999999,"15min":0.36099999999999999},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.43099999999999999,"15min":0.42399999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65900000000000003,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.35999999999999999}]},{"osd":4,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.79300000000000004,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73799999999999999,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.65800000000000003}]},{"osd":5,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.40300000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79200000000000004,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.42799999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86099999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.879,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81399999999999995,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.59199999999999997}]},{"osd":7,"last update":"Thu Apr 25 20:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.622,"15min":0.625},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91100000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84499999999999997,"5min":1.079,"15min":1.079},"last":0.60399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":9078,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:11:50.021 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T20:11:50.337 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T20:11:50.337 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T20:11:50.337 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T20:11:50.337 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T20:11:50.371 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T20:11:50.371 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T20:11:50.371 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T20:11:50.436 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T20:11:50.437 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T20:11:50.437 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T20:11:50.505 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T20:11:50.505 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T20:11:50.505 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T20:11:50.574 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T20:11:50.574 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T20:11:50.574 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T20:11:50.609 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T20:11:50.609 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T20:11:50.609 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T20:11:50.674 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T20:11:50.674 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T20:11:50.674 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T20:11:50.746 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T20:11:50.746 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T20:11:50.746 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T20:11:50.812 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T20:11:50.812 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T20:11:50.813 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T20:11:50.813 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T20:11:50.839 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T20:11:50.839 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T20:11:50.839 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T20:11:50.880 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T20:11:50.880 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T20:11:50.880 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T20:11:50.880 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T20:11:50.891 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T20:11:50.891 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T20:11:50.891 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T20:11:50.942 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T20:11:50.942 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T20:11:50.942 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T20:11:50.954 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T20:11:50.954 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T20:11:50.954 DEBUG:teuthology.orchestra.run.smithi039:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | head -n 1 2024-04-25T20:11:51.011 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi039.front.sepia.ceph.com 2024-04-25T20:11:51.011 DEBUG:teuthology.orchestra.run.smithi039:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T20:11:51.178 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi039.front.sepia.ceph.com 2024-04-25T20:11:51.178 DEBUG:teuthology.orchestra.run.smithi039:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T20:11:51.260 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi039.front.sepia.ceph.com 2024-04-25T20:11:51.260 DEBUG:teuthology.orchestra.run.smithi039:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T20:11:51.311 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi039.front.sepia.ceph.com 2024-04-25T20:11:51.311 DEBUG:teuthology.orchestra.run.smithi039:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T20:11:51.374 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi135.front.sepia.ceph.com 2024-04-25T20:11:51.374 DEBUG:teuthology.orchestra.run.smithi135:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T20:11:51.510 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi135.front.sepia.ceph.com 2024-04-25T20:11:51.510 DEBUG:teuthology.orchestra.run.smithi135:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T20:11:51.573 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi135.front.sepia.ceph.com 2024-04-25T20:11:51.573 DEBUG:teuthology.orchestra.run.smithi135:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T20:11:51.630 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi135.front.sepia.ceph.com 2024-04-25T20:11:51.631 DEBUG:teuthology.orchestra.run.smithi135:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T20:11:51.681 INFO:tasks.ceph:Archiving mon data... 2024-04-25T20:11:51.682 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-04-23_22:16:02-rbd-reef-distro-default-smithi/7670822/data/mon.a.tgz 2024-04-25T20:11:51.682 DEBUG:teuthology.orchestra.run.smithi039:> mktemp 2024-04-25T20:11:51.697 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/tmp.VbZA8MpLbF 2024-04-25T20:11:51.697 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.VbZA8MpLbF 2024-04-25T20:11:51.827 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0666 /tmp/tmp.VbZA8MpLbF 2024-04-25T20:11:51.910 DEBUG:teuthology.orchestra.remote:smithi039:/tmp/tmp.VbZA8MpLbF is 284KB 2024-04-25T20:11:52.006 DEBUG:teuthology.orchestra.run.smithi039:> rm -fr /tmp/tmp.VbZA8MpLbF 2024-04-25T20:11:52.021 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-04-23_22:16:02-rbd-reef-distro-default-smithi/7670822/data/mon.c.tgz 2024-04-25T20:11:52.021 DEBUG:teuthology.orchestra.run.smithi039:> mktemp 2024-04-25T20:11:52.075 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/tmp.2uffIFHowV 2024-04-25T20:11:52.075 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.2uffIFHowV 2024-04-25T20:11:52.216 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0666 /tmp/tmp.2uffIFHowV 2024-04-25T20:11:52.299 DEBUG:teuthology.orchestra.remote:smithi039:/tmp/tmp.2uffIFHowV is 402KB 2024-04-25T20:11:52.394 DEBUG:teuthology.orchestra.run.smithi039:> rm -fr /tmp/tmp.2uffIFHowV 2024-04-25T20:11:52.409 DEBUG:teuthology.misc:Transferring archived files from smithi135:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-04-23_22:16:02-rbd-reef-distro-default-smithi/7670822/data/mon.b.tgz 2024-04-25T20:11:52.410 DEBUG:teuthology.orchestra.run.smithi135:> mktemp 2024-04-25T20:11:52.428 INFO:teuthology.orchestra.run.smithi135.stdout:/tmp/tmp.oFRBE8m6YK 2024-04-25T20:11:52.428 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.oFRBE8m6YK 2024-04-25T20:11:52.590 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 0666 /tmp/tmp.oFRBE8m6YK 2024-04-25T20:11:52.672 DEBUG:teuthology.orchestra.remote:smithi135:/tmp/tmp.oFRBE8m6YK is 402KB 2024-04-25T20:11:52.768 DEBUG:teuthology.orchestra.run.smithi135:> rm -fr /tmp/tmp.oFRBE8m6YK 2024-04-25T20:11:52.783 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T20:11:52.783 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T20:11:52.785 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T20:11:52.846 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-04-25T20:11:52.846 DEBUG:teuthology.orchestra.run.smithi039:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-04-25T20:11:52.849 DEBUG:teuthology.orchestra.run.smithi135:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-04-25T20:11:52.873 INFO:teuthology.orchestra.run.smithi039.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-04-25T20:11:52.912 INFO:teuthology.orchestra.run.smithi135.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-04-25T20:11:52.916 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T20:11:52.918 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-23_22:16:02-rbd-reef-distro-default-smithi/7670822/remote/smithi039/crash 2024-04-25T20:11:52.919 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T20:11:52.943 DEBUG:teuthology.misc:Transferring archived files from smithi135:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-23_22:16:02-rbd-reef-distro-default-smithi/7670822/remote/smithi135/crash 2024-04-25T20:11:52.944 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T20:11:52.979 INFO:tasks.ceph:Compressing logs... 2024-04-25T20:11:52.980 DEBUG:teuthology.orchestra.run.smithi039:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T20:11:52.987 DEBUG:teuthology.orchestra.run.smithi135:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T20:11:53.837 INFO:tasks.ceph:Archiving logs... 2024-04-25T20:11:53.837 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-23_22:16:02-rbd-reef-distro-default-smithi/7670822/remote/smithi039/log 2024-04-25T20:11:53.838 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T20:11:54.200 DEBUG:teuthology.misc:Transferring archived files from smithi135:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-23_22:16:02-rbd-reef-distro-default-smithi/7670822/remote/smithi135/log 2024-04-25T20:11:54.201 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T20:11:54.319 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T20:11:54.347 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T20:11:54.347 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T20:11:54.350 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T20:11:54.423 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T20:11:54.432 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T20:11:54.432 DEBUG:teuthology.orchestra.run.smithi039:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T20:11:54.435 DEBUG:teuthology.orchestra.run.smithi135:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T20:11:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:bash: line 1: ntpq: command not found 2024-04-25T20:11:54.449 INFO:teuthology.orchestra.run.smithi135.stderr:bash: line 1: ntpq: command not found 2024-04-25T20:11:54.451 INFO:teuthology.orchestra.run.smithi039.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T20:11:54.451 INFO:teuthology.orchestra.run.smithi039.stdout:=============================================================================== 2024-04-25T20:11:54.453 INFO:teuthology.orchestra.run.smithi039.stdout:^* hv01.front.sepia.ceph.com 3 6 377 15 +596us[ +603us] +/- 41ms 2024-04-25T20:11:54.453 INFO:teuthology.orchestra.run.smithi039.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 15 -1062us[-1054us] +/- 78ms 2024-04-25T20:11:54.453 INFO:teuthology.orchestra.run.smithi039.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 -118us[ -111us] +/- 83ms 2024-04-25T20:11:54.453 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:11:54.454 INFO:teuthology.orchestra.run.smithi135.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T20:11:54.454 INFO:teuthology.orchestra.run.smithi135.stdout:=============================================================================== 2024-04-25T20:11:54.454 INFO:teuthology.orchestra.run.smithi135.stdout:^* hv01.front.sepia.ceph.com 3 6 377 15 +596us[ +600us] +/- 41ms 2024-04-25T20:11:54.454 INFO:teuthology.orchestra.run.smithi135.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 15 -1062us[-1058us] +/- 78ms 2024-04-25T20:11:54.454 INFO:teuthology.orchestra.run.smithi135.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 17 -119us[ -114us] +/- 83ms 2024-04-25T20:11:54.454 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:11:54.455 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T20:11:54.463 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T20:11:54.464 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T20:11:54.472 DEBUG:teuthology.orchestra.run.smithi039:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T20:11:54.563 DEBUG:teuthology.orchestra.run.smithi135:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T20:11:54.628 DEBUG:teuthology.orchestra.run.smithi039:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T20:11:54.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:11:54.656 DEBUG:teuthology.orchestra.run.smithi135:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T20:11:54.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:11:54.683 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T20:11:54.692 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T20:11:54.701 INFO:teuthology.task.internal:Duration was 558.837574 seconds 2024-04-25T20:11:54.701 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T20:11:54.709 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T20:11:54.709 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T20:11:54.712 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T20:11:54.747 INFO:teuthology.orchestra.run.smithi039.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T20:11:54.758 INFO:teuthology.orchestra.run.smithi135.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T20:11:55.101 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T20:11:55.102 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi039.front.sepia.ceph.com 2024-04-25T20:11:55.102 DEBUG:teuthology.orchestra.run.smithi039:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T20:11:55.162 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi135.front.sepia.ceph.com 2024-04-25T20:11:55.162 DEBUG:teuthology.orchestra.run.smithi135:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T20:11:55.182 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T20:11:55.182 DEBUG:teuthology.orchestra.run.smithi039:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T20:11:55.205 DEBUG:teuthology.orchestra.run.smithi135:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T20:11:55.247 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T20:11:55.247 DEBUG:teuthology.orchestra.run.smithi039:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T20:11:55.271 DEBUG:teuthology.orchestra.run.smithi135:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T20:11:55.392 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T20:11:55.407 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T20:11:55.408 DEBUG:teuthology.orchestra.run.smithi039:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T20:11:55.432 DEBUG:teuthology.orchestra.run.smithi135:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T20:11:55.456 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T20:11:55.466 DEBUG:teuthology.orchestra.run.smithi039:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T20:11:55.476 DEBUG:teuthology.orchestra.run.smithi135:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T20:11:55.496 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = core 2024-04-25T20:11:55.518 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern = core 2024-04-25T20:11:55.536 DEBUG:teuthology.orchestra.run.smithi039:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T20:11:55.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:11:55.569 DEBUG:teuthology.orchestra.run.smithi135:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T20:11:55.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:11:55.590 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T20:11:55.601 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T20:11:55.601 DEBUG:teuthology.misc:Transferring archived files from smithi039:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-23_22:16:02-rbd-reef-distro-default-smithi/7670822/remote/smithi039 2024-04-25T20:11:55.602 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T20:11:55.643 DEBUG:teuthology.misc:Transferring archived files from smithi135:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-23_22:16:02-rbd-reef-distro-default-smithi/7670822/remote/smithi135 2024-04-25T20:11:55.644 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T20:11:55.676 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T20:11:55.676 DEBUG:teuthology.orchestra.run.smithi039:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T20:11:55.680 DEBUG:teuthology.orchestra.run.smithi135:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T20:11:55.725 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T20:11:55.735 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T20:11:55.736 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T20:11:55.744 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T20:11:55.744 DEBUG:teuthology.orchestra.run.smithi039:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T20:11:55.747 DEBUG:teuthology.orchestra.run.smithi135:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T20:11:55.759 INFO:teuthology.orchestra.run.smithi039.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 20:11 /home/ubuntu/cephtest 2024-04-25T20:11:55.779 INFO:teuthology.orchestra.run.smithi135.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 20:11 /home/ubuntu/cephtest 2024-04-25T20:11:55.780 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T20:11:55.791 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T20:11:55.817 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/journal} duration: 558.8375737667084 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-25T20:11:55.817 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T20:11:55.907 INFO:teuthology.run:pass